Commit Graph

425 Commits

Author SHA1 Message Date
Taneli Hukkinen
be16cfa035
Get click types from main repo (#2344)
Click types have been moved to click repo itself. See pallets/click#1856

I've had some issues with typeshed types being outdated in another project
so might be good to avoid that here.

Commit history before merge:

* Get `click` types from main repo
* Fix mypy errors
* Require click v8 for type annotations
* Update Pipfile
2021-06-22 11:58:49 -04:00
jack1142
52f402dcfb
Add EOF and trailing whitespace fixer to pre-commit config (#2330) 2021-06-13 10:22:46 -07:00
Bryan Bugyi
e2fd914dc1
Fix internal error when FORCE_OPTIONAL_PARENTHESES feature is enabled (#2332)
Fixes #2313.
2021-06-13 10:20:50 -07:00
Cooper Lees
aa31a117b1
Add STDIN test to primer (#2315)
* Add STDIN test to primer

- Check that out STDIN black support stays working
- Add asyncio.subprocess STDIN pip via communicate
- We just check we format python code from primer's `lib.py`

Fixes #2310
2021-06-10 21:06:50 -07:00
jack1142
62402a3261
Support named escapes (\N{...}) in string processing (#2319)
Co-authored-by: Felix Hildén <felix.hilden@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2021-06-09 12:29:32 -07:00
Richard Si
00e7e12a3a
Regression fix: leave R prefixes capitalization alone (#2285)
`black.strings.get_string_prefix` used to lowercase the extracted
prefix before returning it. This is wrong because 1) it ignores the
fact we should leave R prefixes alone because of MagicPython, and 2)
there is dedicated prefix casing handling code that fixes issue 1.
`.lower` is too naive.

This was originally fixed in 20.8b0, but was reintroduced since 21.4b0.

I also added proper prefix normalization for docstrings by using the
`black.strings.normalize_string_prefix` helper.

Some more test strings were added to make sure strings with capitalized
prefixes aren't treated differently (actually happened with my original
patch, Jelle had to point it out to me).
2021-06-08 17:46:09 -07:00
Felix Hildén
a9eab85f22
Mention comment non-processing in documentation (#2306)
This commit adds a short section discussing the non-processing of docstrings
besides spacing improvements, mentions comment moving and links to the
AST equivalence discussion. I also added a simple spacing test for good
measure.

Commit history before merge:

* Mention comment non-processing in documentation, add spacing test
* Mention special cases for comment spacing
* Add all special cases, improve wording
2021-06-08 17:57:23 -04:00
Sergey Vartanov
40fae18134
Possible fix for issue with indentation and fmt: skip (#2281)
Not sure the fix is right.  Here is what I found: issue is connected
with line

    first.prefix = prefix[comment.consumed :]

in `comments.py`.  `first.prefix` is a prefix of the line, that ends
with `# fmt: skip`, but `comment.consumed` is the length of the
`"  # fmt: skip"` string.  If prefix length is greater than 14,
`first.prefix` will grow every time we apply formatting.

Fixes #2254
2021-06-08 14:37:34 -07:00
Bryan Bugyi
99b68e59ce
Fix incorrect custom breakpoint indices when string group contains fake f-strings (#2311)
Fixes #2293
2021-06-07 07:03:39 -07:00
Bryan Bugyi
6380b9f2f6
Account for += assignment when deciding whether to split string (#2312)
Fixes #2294
2021-06-07 07:01:57 -07:00
Felix Hildén
a2b5ba2a3a
Add option to require a specific version to be running (#2300)
Closes #1246: This PR adds a new option (and automatically a toml entry, hooray for existing configuration management 🎉) to require a specific version of Black to be running.

For example: `black --required-version 20.8b -c "format = 'this'"`

Execution fails straight away if it doesn't match `__version__`.
2021-06-03 13:09:41 -07:00
Hassan Abouelela
7567cdf3b4
Code Flag Options (#2259)
Properly handles the diff, color, and fast option when black is run with
 the `--code` option.

Closes #2104, closes #1801.
2021-06-01 18:55:21 -07:00
Bryan Bugyi
a4e35b3149
Correct max string length calculation when there are string operators (#2292)
PR #2286 did not fix the edge-cases (e.g. when the string is just long
enough to cause a line to be 89 characters long). This PR corrects that
mistake.
2021-05-31 17:57:23 -07:00
Bryan Bugyi
199e3eb76b
Fix regular expression that black uses to identify f-expressions (#2287)
Fixes #1469
2021-05-30 15:34:33 -07:00
Bryan Bugyi
4ca4407b4a
Make sure to split lines that start with a string operator (#2286)
Fixes #2284
2021-05-30 23:41:03 +02:00
Bryan Bugyi
eec44f5977
Fix --experiemental-string-processing crash when matching parens not found (#2283)
Fixes #2271
2021-05-30 12:32:28 -07:00
Mark Bell
92f20d7f84
Removed adding a space into empty docstrings. (#2249)
Resolves #2168 by disabling the insertion of a " " when the docstring is entirely empty.

Note that this PR is focussed only on the case of empty docstrings. In particular this does not make any changes to the behaviour that a " " is inserted if a non-empty docstring begins with the quoting character. That is, black still prefers:

    """ "something" """

to:

    """"something" """

and that:

    """"Something""""

is not a legal docstring.
2021-05-25 15:43:28 -07:00
Hadi Alqattan
b8450b9fae
Fix: black only respects the root gitignore. (#2225)
Commit history before merge:

Black now respects .gitignore files in all levels, not only root/.gitignore file
(apply .gitignore rules like git does).

* Fix: typo
* Fix: respect .gitignore files in all levels.
* Add: CHANGELOG note.
* Fix: TypeError: unsupported operand type(s) for +: 'NoneType' and 'PathSpec'
* Update docs.
* Fix: no parent .gitignore
* Add a comment since the if expression is a bit hard to understand
* Update tests - conver no parent .gitignore case.
* Use main's Pipfile.lock instead

  The original changes in Pipfile.lock are whitespace only. The changes
  turned the JSON's file indentation from 4 to 2. Effectively this
  happened: `json.dumps(json.loads(old_pipfile_lock), indent=2) + "\n"`.

  Just using main's Pipfile.lock instead of undoing the changes because
  1) I don't know how to do that easily and quickly, and 2) there's a
  merge conflict.

  Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>

* Merge remote-tracking branch 'upstream/main' into i1730 …
  
  conflicts for days ay?
2021-05-16 13:51:27 -04:00
Christian Clauss
445f094f1f
Use codespell to find typos (#2228) 2021-05-13 10:28:41 -07:00
Richard Si
94a0b07dbe
Remove useless flake8 config + test support code (#2221)
We've depended on Click 7.x ever since we broke CI systems across the
world (oops lol) and flake8-mypy was purged a fair bit back: #1867

Also remove the primer tests import in tests/test_black.py because it's
annoying when just trying to actually target tests/test_black.py tests.
`pytest -k test_black.py` doesn't do what you expect due to that import.
2021-05-11 14:09:33 -04:00
Richard Si
036bea4aa0
Speed up tests even more (#2205)
There's three optimizations in this commit:

1. Don't check if Black's output is stable or equivalant if no changes
   were made in the first place. It's not like passing the same code
   (for both source and actual) through black.assert_equivalent or
   black.assert_stable is useful. It's not a big deal for the smaller
   tests, but it eats a lot of time in tests/test_format.py since
   its test cases are big. This is also closer to how Black works IRL.

2. Use a smaller file for `test_root_logger_not_used_directly` since
   the logging it's checking happens during blib2to3's startup so the
   file doesn't really matter.

3. If we're checking a file is formatting (i.e. test_source_is_formatted)
   don't run Black over it again with `black.format_file_in_place`.
   `tests/test_format.py::TestSimpleFormat.check_file` is good enough.
2021-05-08 11:34:25 +02:00
Łukasz Langa
f2ea461e9e
Refactor src/black/__init__.py into many files (#2206)
* Move string-related utility to functions to strings.py, const.py
* Move Leaf/Node-related functionality to nodes.py
* Move comment-related functions to comments.py
* Move caching to cache.py and Mode/TargetVersion/Feature to mode.py
* Move some leftover functions to nodes.py, comments.py, strings.py
* Add missing files to source list for test runs
* Move line-related functionality into lines.py, brackets into brackets.py
* Move transformers to trans.py
* Move file handling, output, parsing, concurrency, debug, and report
* Move two more functions to nodes.py
* Add CHANGES
* Add numeric.py
* Add linegen.py
* More docstrings
* Include new files in tests

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2021-05-08 11:29:47 +02:00
Łukasz Langa
d0e06b53b0
Mark blackd tests with the blackd optional marker (#2204)
This is a follow-up of #2203 that uses a pytest marker instead of a bunch of
`skipUnless`.  Similarly to the Python 2 tests, they are running by default and
will crash on an unsuspecting contributor with missing dependencies.  This is
by design, we WANT contributors to test everything.  Unless we actually don't
and then we can run:

  pytest --run-optional=no_blackd

Relatedly, bump required aiohttp to 3.6.0 at least to get rid of expected
failures on Python 3.8 (see 6b5eb7d465).
2021-05-07 16:33:36 +02:00
Łukasz Langa
e4b4fb02b9
Use optional tests for "no_python2" to simplify local testing (#2203) 2021-05-07 15:03:13 +02:00
Kaleb Barrett
1fe2efd857
Do not use gitignore if explicitly passing excludes (#2170)
Closes #2164.

Changes behavior of how .gitignore is handled. With this change, the rules in .gitignore are only used as a fallback if no exclusion rule is explicitly passed on the command line or in pyproject.toml. Previously they were used regardless if explicit exclusion rules were specified, preventing any overriding of .gitignore rules.

Those that depend only on .gitignore for their exclusion rules will not be affected. Those that use both .gitignore and exclude will find that exclude will act more like actually specifying exclude and not just another extra-excludes. If the previous behavior was desired, they should move their rules from exclude to extra-excludes.
2021-05-07 14:54:21 +02:00
KotlinIsland
204f76e0c0
add test configurations that don't contain python2 optional install (#2190)
add test for negative scenario: formatting python2 code
tag python2 only tests

Co-authored-by: KotlinIsland <kotlinisland@users.noreply.github.com>
2021-05-04 10:47:22 +02:00
Richard Si
e42f9921e2
Detect '@' dotted_name '(' ')' NEWLINE as a simple decorator (#2182)
Previously the RELAXED_DECORATOR detection would be falsely True on that
example. The problem was that an argument-less parentheses pair didn't
pass the `is_simple_decorator_trailer` check even it should. OTOH a
parentheses pair containing an argument or more passed as expected.
2021-05-04 10:46:46 +02:00
Cooper Lees
a18c7bc099
primer: Add --no-diff option (#2187)
- Allow runs with no code diff output
- This is handy for reducing output to see which file is erroring

Test:
- Edit config for 'channels' to expect no changes and run with `--no-diff` and see no diff output
2021-05-04 10:44:40 +02:00
Bryan Forbes
35e8d1560d
Set is_pyi if stdin_filename ends with .pyi (#2169)
Fixes #2167

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2021-05-02 14:48:54 +02:00
Richard Si
b55ea63ff4
Stop stripping parens in even more illegal spots (#2148)
We're only fixing them so fuzzers don't yell at us when we break "valid"
code. I mean "valid" because some of the examples aren't even accepted by
Python.
2021-04-26 22:26:43 +02:00
Łukasz Langa
eaa337f176
Add more tests for fancy whitespace (#2147) 2021-04-26 20:24:06 +02:00
Jelle Zijlstra
557b54aa60
Fix crash on docstrings ending with "\ " (#2142)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-04-26 19:42:16 +02:00
Richard Si
db30456916
Don't strip parens in assert / return with assign expr (#2143)
Black would previously strip the parenthesis away from statements like this these ones:

    assert (spam := 12 + 1)
    return (cheese := 1 - 12)

Which happens to be invalid code. Now before making the parenthesis invisible, Black
checks if the assignment expression's parent is an assert stamtment, aborting if True.

Raise, yield, and await are already handled fine.

I added a bunch of test cases from the PEP defining asssignment expressions (PEP 572).
2021-04-26 08:28:42 -07:00
Jelle Zijlstra
0a833b4b14
fix magic comma and experimental string cache flags (#2131)
* fix magic comma and experimental string cache flags

* more changelog

* Update CHANGES.md

Co-authored-by: Cooper Lees <me@cooperlees.com>

* fix tests

Co-authored-by: Cooper Lees <me@cooperlees.com>
2021-04-26 07:46:48 +02:00
Łukasz Langa
8672af35f0
Work around stability errors due to optional trailing commas (#2126)
Optional trailing commas put by Black become magic trailing commas on another
pass of the tool.  Since they are influencing formatting around optional
parentheses, on rare occasions the tool changes its mind in terms of putting
parentheses or not.

Ideally this would never be the case but sadly the decision to put optional
parentheses or not (which looks at pre-existing "magic" trailing commas) is
happening around the same time as the decision to put an optional trailing
comma.  Untangling the two proved to be impractically difficult.

This shameful workaround uses the fact that the formatting instability
introduced by magic trailing commas is deterministic: if the optional trailing
comma becoming a pre-existing "magic" trailing comma changes formatting, the
second pass becomes stable since there is no variable factor anymore on pass 3,
4, and so on.

For most files, this will introduce no performance penalty since `--safe` is
already re-formatting everything twice to ensure formatting stability.  We're
using this result and if all's good, the behavior is equivalent.  If there is
a difference, we treat the second result as the binding one, and check its
sanity again.
2021-04-25 20:15:54 +02:00
Łukasz Langa
773e4a22d5
Revert "Use lowercase hex numbers fixes #1692 (#1775)"
This reverts commit 7d032fa848.
2021-04-25 19:13:23 +02:00
Felix Hildén
368f043f13
Document experimental string processing and docstring indentation (#2106) 2021-04-22 10:37:27 -07:00
CiderMan
5316bbff0e
Handle Docstrings as bytes + strip all whitespace (#2037)
(fixes #1844, fixes #1923, fixes #1851, fixes #2002, fixes #2103)
2021-04-22 08:40:51 -07:00
Mark Bell
1fc3215e8c
Make black remove leading and trailing spaces from one-line docstrings (#1740)
Fixes #1738. Fixes #1812.

Previously, Black removed leading and trailing spaces in multiline docstrings but failed to remove them from one-line docstrings.
2021-04-22 08:23:41 -07:00
Pierre Sassoulas
d960d5d238
Remove NBSP at the beginning of comments (#2092)
Closes #2091
2021-04-11 14:41:22 -07:00
Harish Rajagopal
9451c57d1c
Support for top-level user configuration (#1899)
* Added support for top-level user configuration

At the user level, a TOML config can be specified in the following locations:
* Windows: ~\.black
* Unix-like: $XDG_CONFIG_HOME/black (~/.config/black fallback)

Instead of changing env vars for the entire black-primer process, they
are now changed only for the black subprocess, using a tmpdir.
2021-04-01 18:39:18 +02:00
Joshua Cannon
e3c71c3a47
Turn test_regex into a click callback (#2016)
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
2021-03-02 17:21:50 -08:00
Joshua Cannon
beecd6fd0a
Add --extend-exclude parameter (#2005)
Look ma! I contribute to open source!

Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
2021-03-01 14:07:36 -08:00
Rishav Kundu
858225d34d
Strip redundant parentheses from assignment exprs (#1906)
Fixes #1656
2021-02-27 17:20:23 -08:00
tpilewicz
b06cd15666
Wrap arithmetic and binary arithmetic expressions in invisible parentheses (#2001) 2021-02-24 03:56:56 -08:00
Paul "TBBle" Hampson
cd4295dd98
Indicate that a final newline was added in --diff (#1897) (#1897)
Fixes: #1662

Work-around for https://bugs.python.org/issue2142

The test has to slightly mess with its input data, because the utility
functions default to ensuring the test data has a final newline, which
defeats the point of the test.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-02-21 22:43:23 -08:00
Sagi Shadur
6a105e019f
Add "# fmt: skip" directive to black (#1800)
Fixes #1162
2021-02-15 08:02:48 -08:00
James Addison
b8c1020b52
Stability fixup: interaction between newlines and comments (#1975)
* Add test case to illustrate the issue

* Accept carriage returns as valid separators while enumerating comments

Without this acceptance, escaped multi-line statments that use carriage returns will not be counted into the 'ignored_lines' variable since the emitted line values will end with a CR and not an escape character.  That leads to comments associated with the line being incorrectly labeled with the STANDALONE_COMMENT type, affecting comment placement and line space management.

* Remove comment linking to ephemeral build log
2021-02-11 12:11:42 -08:00
Anthony Sottile
3fca540d05
speed up cache by approximately 42x by avoiding pathlib (#1953) 2021-02-04 13:03:42 -08:00
Shantanu
692c0f50d9
Add --skip-magic-trailing-comma (#1824) 2021-01-17 16:59:06 -08:00
Bryan Bugyi
e912c7ff54
Fix INTERNAL ERROR caused by removing parens from pointless string (#1888)
Fixes #1846.
2020-12-28 12:30:23 -08:00
Richard Si
4d03716eae
Allow same RHS expressions in annotated assignments as in regular assignments (#1835) 2020-11-24 09:39:25 +00:00
Thiago Bellini Ribeiro
dea81b7ad5
Provide a stdin-filename to allow stdin to respect force-exclude rules (#1780)
* Provide a stdin-filename to allow stdin to respect exclude/force-exclude rules

This will allow automatic tools to enforce the project's
exclude/force-exclude rules even if they pass the file through stdin to
update its buffer.

This is a similar solution to --stdin-display-name in flake8.

* Update src/black/__init__.py

Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>

* --stdin-filename should only respect --exclude-filename

* Update README with the new --stdin-filename option

* Write some tests for the new stdin-filename functionality

* Apply suggestions from code review

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

* Force stdin output when we asked for stdin even if the file exists

* Add an entry in the changelog regarding --stdin-filename

* Reduce disk reads if possible

Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>

* Check for is_stdin and p.is_file before checking for p.is_dir()

Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2020-11-13 07:26:07 -08:00
Casper Weiss Bang
7d032fa848
Use lowercase hex numbers fixes #1692 (#1775)
* Made hex lower case

* Refactored numeric formatting section

* Redid some refactoring and removed bloat

* Removed additions from test_requirements.txt

* Primer now expects expected changes

* Undid some refactoring

* added to changelog

* Update src/black/__init__.py

Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>

Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>
Co-authored-by: Cooper Lees <me@cooperlees.com>
2020-11-13 07:25:17 -08:00
Justin Prieto
1d8b4d766d
Correctly handle inline tabs in docstrings (#1810)
The `fix_docstring` function expanded all tabs, which caused a
difference in the AST representation when those tabs were inline and not
leading. This changes the function to only expand leading tabs so inline
tabs are preserved.

Fixes #1601.
2020-11-09 11:58:23 -08:00
Bryan Bugyi
edf1c9dc0f
Fix bug which causes f-expressions to be split (#1809)
Closes #1807.
2020-11-06 16:17:23 -08:00
Bryan Bugyi
6c3f818185
Fix bug where black tries to split string on escaped space (#1799)
Closes #1505.
2020-10-31 10:42:36 -07:00
Sagi Shadur
e6cd10e761
Extract formatting tests (#1785)
* Update test versions

* Use parametrize to remove tests duplications

* Extract sources format tests

* Fix mypy errors

* Fix .travis.yml
2020-10-30 08:12:04 -07:00
Sagi Shadur
407052724f
Switch to pytest and tox (#1763)
* Add venv to .gitignore

* Use tox to run tests

* Make fuzz run in tox

* Split tests files

* Fix import error
2020-10-19 10:35:26 -07:00
QuentinSoubeyran
6dddbd7241
PEP 614 support (#1717) 2020-09-19 20:33:10 +02:00
Richard Si
c0a8e42243
Fix empty line handling when formatting typing stubs (#1646)
Black used to erroneously remove all empty lines between non-function
code and decorators when formatting typing stubs. Now a single empty
line is enforced.

I chose for putting empty lines around decorated classes that have empty
bodies since removing empty lines around such classes would cause a
formatting issue that seems to be impossible to fix.

For example:

```
class A: ...
@some_decorator
class B: ...
class C: ...
class D: ...

@some_other_decorator
def foo(): -> None: ...
```

It is easy to enforce no empty lines between class A, B, and C.
Just return 0, 0 for a line that is a decorator and precedes an stub
class. Fortunately before this commit, empty lines after that class
would be removed already.

Now let's look at the empty line between class D and function foo. In
this case, there should be an empty line there since it's class code next
to function code. The problem is that when deciding to add X empty lines
before a decorator, you can't tell whether it's before a class or a
function. If the decorator is before a function, then an empty line
is needed, while no empty lines are needed when the decorator is
before a class.

So even though I personally prefer no empty lines around decorated
classes, I had to go the other way surrounding decorated classes with
empty lines.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2020-09-10 13:21:37 -07:00
Bryan Bugyi
cd055efd7d
Fix unstable subscript assignment string wrapping (#1678)
Fixes #1598
2020-09-10 09:24:01 -07:00
Bryan Bugyi
6284953d07
Fix crash on assert and parenthesized % format (fixes #1597, fixes #1605) (#1681) 2020-09-06 09:15:40 -07:00
Bryan Bugyi
7bca930ca3
Fix crash on concatenated string + comment (fixes #1596) (#1677)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2020-09-06 08:02:57 -07:00
Bryan Bugyi
e3ccabb23c
Fix unstable formatting on string split + % formatting (#1680)
Fixes #1595
2020-09-05 17:24:00 -07:00
Tom Saunders
6b5753a417
Handle .COLOR_DIFF in the same way as .DIFF (#1673) 2020-09-05 12:15:28 -07:00
Richard Si
1d2d7264ec
Fix incorrect space before colon in if/while stmts (#1655)
* Fix incorrect space before colon in if/while stmts

Previously Black would format this code

```
if (foo := True):
	print(foo)
```

as

```
if (foo := True) :
	print(foo)
```

adding an incorrect space after the RPAR. Buggy code in the
normalize_invisible_parens function caused the colon to be wrapped in
invisible parentheses. The LPAR of that pair was then prefixed with a
single space at the request of the whitespace function.

This commit fixes the accidental skipping of a pre-condition check
which must return True before parenthesis normalization of a specific
child Leaf or Node can happen. The pre-condition check being skipped
was why the colon was wrapped in invisible parentheses.

* Add an entry in CHANGES.md
2020-08-31 14:20:05 -07:00
mbarkhau
2b75f8870e
fix 1631 and add test (#1641) 2020-08-27 04:47:59 -07:00
Richard Si
7fe19fac5b Fix multiline docstring quote normalization
The quotes of multiline docstrings are now only normalized when string
normalization is off, instead of the string normalization setting being
ignored and the quotes being *always* normalized.

I had to make a new test case and data file since the current pair for
docstrings only worked when there is no formatting difference between the
formatting results with string normalization on and off. I needed to add
tests for when there *are* differences between the two. So I split
test_docstring's test code when string normalization is disabled into a
new test case along with a new data file.
2020-08-27 01:59:41 +02:00
Łukasz Langa
ceeb1d9a2e Add expected failure tests with the unstable formattings 2020-08-26 16:55:05 +02:00
Łukasz Langa
9270a10f6f Improve docstring re-indentation handling
This addresses a few crashers, namely:

* producing non-equivalent code due to mangling escaped newlines,

* invalid hugging quote characters in the docstring body to the docstring outer
  triple quotes (causing a quadruple quote which is a syntax error),

* lack of handling for docstrings that start on the same line as the `def`, and

* invalid stripping of outer triple quotes when the docstring contained
  a string prefix.

As a bonus, tests now also run when string normalization is disabled.
2020-08-25 23:14:39 +02:00
Łukasz Langa
586d24236e Address pre-existing trailing commas when not in the rightmost bracket pair
This required some hackery.  Long story short, we need to reuse the ability to
omit rightmost bracket pairs (which glues them together and splits on something
else instead), for use with pre-existing trailing commas.

This form of user-controlled formatting is brittle so we have to be careful not
to cause a scenario where Black first formats code without trailing commas in
one way, and then looks at the same file with pre-existing trailing commas
(that it itself put on the previous run) and decides to format the code again.

One particular ugly edge case here is handling of optional parentheses.  In
particular, the long-standing `line_length=1` hack got in the way of
pre-existing trailing commas and had to be removed.  Instead, a more
intelligent but costly solution was put in place: a "second opinion" if the
formatting that omits optional parentheses ended up causing lines to be too
long.  Again, for efficiency purposes, Black reuses Leaf objects from blib2to3
and modifies them in place, which was invalid for having two separate
formattings.  Line cloning was used to mitigate this.

Fixes #1619
2020-08-25 22:10:05 +02:00
Łukasz Langa
d46268cd67
Run trailing comma tests with TargetVersion.PY38 2020-08-24 21:33:51 +02:00
Łukasz Langa
292bceb9fd
Add more trailing comma test variants 2020-08-24 18:48:11 +02:00
Łukasz Langa
bc71685d22 Open file explicitly with UTF-8 so it works on Windows, too 2020-08-21 16:45:30 +02:00
Łukasz Langa
788268bc39 Re-implement magic trailing comma handling:
- when a trailing comma is specified in any bracket pair, that signals to Black
  that this bracket pair needs to be always exploded, e.g. presented as "one
  item per line";

- this causes some changes to previously formatted code that erroneously left
  trailing commas embedded into single-line expressions;

- internally, Black needs to be able to identify trailing commas that it put
  itself compared to pre-existing trailing commas. We do this by using/abusing
  lib2to3's `was_checked` attribute.  It's True for internally generated
  trailing commas and False for pre-existing ones (in fact, for all
  pre-existing leaves and nodes).

Fixes #1288
2020-08-21 16:45:30 +02:00
Jelle Zijlstra
e5bb92f53c
Disable string splitting/merging by default (#1609)
* put experimental string stuff behind a flag
* update tests
* don't need an output section if it's the same as the input
* Primer: Expect no formatting changes in attrs, hypothesis and poetry with --experimental-string-processing off

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2020-08-20 14:23:28 +02:00
David Szotten
820f38708f
fix unary op detection (#1600) 2020-08-14 09:17:56 -07:00
David Szotten
d1ad8730e3
don't strip brackets before lsqb (#1575) (#1590)
if the string contains a PERCENT, it's not safe to remove brackets that
follow and operator with the same or higher precedence than PERCENT
2020-08-13 19:20:46 -07:00
Jelle Zijlstra
5e1f620af7
fix some docstring crashes (#1593)
Allow removing some trailing whitespace
2020-08-13 16:40:45 -07:00
Richard Si
97c11f22aa
Make --exclude only apply to recursively found files (#1591)
Ever since --force-exclude was added, --exclude started to touch files
that were given to Black through the CLI too. This is not documented
behaviour and neither expected as --exclude and --force-exclude now
behave the same!

Before this commit, get_sources() when encountering a file that was passed
explicitly through the CLI would pass a single Path object list to
gen_python_files(). This causes bad behaviour since that function
doesn't treat the exclude and force_exclude regexes differently. Which
is fine for recursively found files, but *not* for files given through
the CLI.

Now when get_sources() iterates through srcs and encounters
a file, it checks if the force_exclude regex matches, if not, then the
file will be added to the computed sources set.

A new function had to be created since before you can do regex matching,
the path must be normalized. The full process of normalizing the path is
somewhat long as there is special error handling. I didn't want to
duplicate this logic in get_sources() and gen_python_files() so that's
why there is a new helper function.
2020-08-12 20:07:19 -07:00
Lihu Ben-Ezri-Ravin
2471b9256d
Find project root correctly (#1518)
Ensure root dir is a common parent of all inputs
Fixes #1493
2020-06-24 10:09:07 +01:00
Richard Si
6ebdc5a644
Fix toml parsing and bump toml from 0.10.0 to 0.10.1 (#1501)
* Bump toml from 0.10.0 to 0.10.1 to fix a bug

* Add tests for TOML parsing and reading

* Fix configuration bug affecting vim plugin

The vim plugin directly calls parse_pyproject and skips the Click processing
, but parse_pyproject assumed that it would only be used before Click processing
and therefore made the config values click friendly. This moves the "make the values
click friendly processing" into read_pyproject_toml which is only called by a Click
callback.

* Please mypy and flake8
2020-06-16 11:58:33 -07:00
Adam Williamson
88d12f88a9
expression tests: adjust starred expression for Python 3.9 (#1441) (#1477)
As discussed in #1441, Python 3.9's new parser will not parse
`(*starred)` even using `compile()` with the `PyCF_ONLY_AST`
flag (as `ast.parse()` does), it raises a `SyntaxError`. This
breaks the four tests that use this file with Python 3.9.
Upstream does not consider this to be a bug - see
https://bugs.python.org/issue40848#msg370643 - so we must
adjust the expression. As suggested by @JelleZijlstra, this just
adds a comma, which makes the new parser happy with it (the old
parser is fine with this form also).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-03 15:15:54 -07:00
Cooper Lees
ff6bbd5d96
Capture CalledProcessError for any postitive returncode (#1450)
- Leave logic to still allow for formatting changes to be ignored
- Now just capture the output of any other error that has a > 1 returncode
- Raise on anything else

Test: Add unit test to exercise this new logic
2020-05-22 12:16:31 -07:00
Hugo van Kemenade
a2408b3cb2
black-primer: handle singular and plural in output messages (#1432)
* Handle singular and plural in output messages
2020-05-20 21:03:51 -07:00
Cooper Lees
8acc22f114
Add black-primer unittests (#1426)
* Add black-primer unittests

- Get this tool covered with some decent unittests for all unittests wins
- Have a CLI and lib test class
- Import it from `test_black.py` so we always run tests
- Revert typing asyncio.Queue as Queue[str] so we can work in 3.6
- **mypy**: Until black > 3.6 disallow_any_generics=False for primer code

Test:
- Run tests: `coverage run tests/test_primer.py` or `coverage run -m unittest`
```
(b) cooper-mbp1:black cooper$ coverage report
Name                      Stmts   Miss  Cover
---------------------------------------------
src/black_primer/cli.py      49      8    84%
src/black_primer/lib.py     148     28    81%
tests/test_primer.py        114      1    99%
---------------------------------------------
TOTAL                       311     37    88%
```

* Use ProactorEventLoop for Windows + fix false path for Linux

* Set Windows to use ProactorEventLoop in  to benefit all callers

* sys.platform seems to not having the loop applied - So type ignore and use platform.system() gate

* Have each test loop correctly set to ProactorEventLoop on Windows for < 3.8 too
2020-05-17 12:18:49 -07:00
Hugo van Kemenade
03b8304abd
Update and fix Flake8 (#1424)
* Update pre-commit

* Fix F541 f-string is missing placeholders

* Fix E741 ambiguous variable name 'l'

* Update actions to v2
2020-05-17 07:18:45 -07:00
Jelle Zijlstra
c7da3482c7
fix crashes on docstring whitespace changes (#1417)
Fixes #1415
2020-05-15 20:47:21 -07:00
Cooper Lees
2082a325fd
Refactor black into packages in src/ dir (#1376)
- Move black.py to src/black/__init__.py
- Have setuptools_scm make src/_black_version.py and exclude from git
- Move blackd.py to src/blackd/__init__.py
- Move blib2to3/ to src/
- Update `setup.py`
- Update unittests to pass
  - Mostly path fixing + resolving
- Update CI
  - pre-commit config
  - appveyor + travis

Tested on my mac with python 3.7.5 via:
```
python3 -m venv /tmp/tb3
/tmp/tb3/bin/pip install --upgrade setuptools pip coverage pre-commit
/tmp/tb2/bin/pip install ~/repos/black/
cd ~/repos/black/
/tmp/tb2/bin/coverage run tests/test_black.py
/tmp/tb3/bin/pre-commit run -a
/tmp/tb3/bin/black --help
/tmp/tb3/bin/black ~/repos/ptr/ptr.py
```
2020-05-08 08:50:50 -07:00
Zsolt Dollenstein
7f4b275413
close event loop for all tests (#1394) 2020-05-08 16:27:50 +01:00
Zsolt Dollenstein
703faa3233 Fix mono test 2020-05-08 16:15:31 +01:00
Zsolt Dollenstein
6ea76e9b4f Improve error messages from BlackRunner 2020-05-08 16:15:31 +01:00
Giacomo Tagliabue
89c87d22e7
add --force-exclude argument (#1032)
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2020-05-08 07:47:26 -07:00
Zsolt Dollenstein
67726a7cf3
skip mono test while im working on it 2020-05-08 15:37:08 +01:00
Zsolt Dollenstein
9104ebe5ae
better test for mono executor 2020-05-08 14:57:32 +01:00
Allan Simon
c0a7582e3d
permits black to run in AWS Lambda: (#1141)
AWS Lambda and some other virtualized environment may not permit access
to /dev/shm on Linux and as such, trying to use ProcessPoolExecutor will
fail.

As using parallelism is only a 'nice to have' feature of black, if it fails
we gracefully fallback to a monoprocess implementation, which permits black
to finish normally.

Co-authored-by: Allan Simon <asimon@yolaw.fr>
2020-05-08 15:46:07 +02:00
Douglas Thor
8d6d92aa5b
Add option for printing a colored diff (#1266) 2020-05-08 14:30:10 +01:00
Jon Dufresne
1382eabb3f
Remove deprecated use of 'setup.py test' (#1275)
Since setuptools v41.5.0 (27 Oct 2019), the 'test' command is formally
deprecated and should not be used. Now use unittest as the test entry
point.
2020-05-08 06:23:50 -07:00
williamfzc
92c611cfdf
add bracket check in split_line (#1315) 2020-05-08 06:16:57 -07:00
Alex Vandiver
a4c11a75e1
Re-indent the contents of docstrings (#1053)
* Re-indent the contents of docstrings when indentation changes

Keeping the contents of docstrings completely unchanged when
re-indenting (from 2-space intents to 4, for example) can cause
incorrect docstring indentation:

```
class MyClass:
  """Multiline
  class docstring
  """

  def method(self):
    """Multiline
    method docstring
    """
    pass
```
...becomes:
```
class MyClass:
    """Multiline
  class docstring
  """

    def method(self):
        """Multiline
    method docstring
    """
        pass
```

This uses the PEP 257 algorithm for determining docstring indentation,
and adjusts the contents of docstrings to match their new indentation
after `black` is applied.

A small normalization is necessary to `assert_equivalent` because the
trees are technically no longer precisely equivalent -- some constant
strings have changed.  When comparing two ASTs, whitespace after
newlines within constant strings is thus folded into a single space.

Co-authored-by: Luka Zakrajšek <luka@bancek.net>

* Extract the inner `_v` method to decrease complexity

This reduces the cyclomatic complexity to a level that makes flake8 happy.

* Blacken blib2to3's docstring which had an over-indent

Co-authored-by: Luka Zakrajšek <luka@bancek.net>
Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>
2020-05-08 14:08:15 +01:00
Bryan Bugyi
544ea9c217
Improve String Handling (#1132)
This pull request's main intention is to wraps long strings (as requested by #182); however, it also provides better string handling in general and, in doing so, closes the following issues:

Closes #26
Closes #182
Closes #933
Closes #1183
Closes #1243
2020-05-08 14:56:21 +02:00
otstrel
892eddacd2
Fix for "# fmt: on" with decorators (#1325) 2020-05-08 14:37:17 +02:00
Toby Fleming
7a14a37981
Change exit code to 2 when config file doesn't exist (#1361)
Fixes #1360, where an invalid config file causes a return/exit code of 1. This
change means this case is caught earlier, treated like any other bad
parameters, and results in an exit code of 2.

Co-authored-by: Toby Fleming <tobywf@users.noreply.github.com>
2020-04-30 08:47:52 +01:00
Rémi Verschelde
959848c176
Fix --diff output when encountering EOF (#1328)
`split("\n")` includes a final empty element `""` if the final line
ends with `\n` (as it should for POSIX-compliant text files), which
then became an extra `"\n"`.

`splitlines()` solves that, but there's a caveat, as it will split
on other types of line breaks too (like `\r`), which may not be
desired.

Fixes #526.
2020-04-04 22:02:57 -07:00
Shantanu
6d8b90167b
string prefixes: don't normalise capital R-strings (#1271)
Resolves #1244

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2020-03-03 14:55:14 +01:00
Vlad Emelianov
be49ac72a0 Support py38-style starred expressions in return statement (#1121) 2020-01-18 07:21:46 -08:00
kyle hausmann
a02829bea1 Use conditional case for diff reports (#1226)
When --diff flag is used, black will now use the
conditional case in the Report output: eg "would
be reformatted"
2020-01-18 07:13:15 -08:00
Michael J. Sullivan
4b449e7471 Fix unstable formatting with some # type: ignores (#1113)
`type: ignore` shouldn't block collapsing a line, since it will still
apply fine to the merged line. This prevents an issue where a reformat
causes it to shift lines and then be merged on a subsequent pass.

There is a downside to this, which is that it can cause a `type:
ignore` to apply to more code than was originally intended. There
might be a way to apply this in a more limited situation, but I'm not
sure what it is.

Fixes #1061.
2019-11-25 14:16:00 -08:00
Michael J. Sullivan
3e60f6d454 Support compilation with mypyc (#1009)
* Make most of blib2to3 directly typed and mypyc-compatible

This used a combination of retype and pytype's merge-pyi to do the
initial merges of the stubs, which then required manual tweaking to
make actually typecheck and work with mypyc.

Co-authored-by: Sanjit Kalapatapu <sanjitkal@gmail.com>
Co-authored-by: Michael J. Sullivan <sully@msully.net>

* Make black able to compile and run with mypyc

The changes made fall into a couple categories:
 * Fixing actual type mistakes that slip through the cracks
 * Working around a couple mypy bugs (the most annoying of which being
   that we need to add type annotations in a number of places where
   variables are initialized to None)

Co-authored-by: Sanjit Kalapatapu <sanjitkal@gmail.com>
Co-authored-by: Michael J. Sullivan <sully@msully.net>
2019-10-30 07:29:29 -07:00
Lawrence Chan
23fec8b0f7 Fix fmt on/off when multiple exist in leaf prefix (#1086)
The old behavior would detect the existence of a `# fmt: on` in a leaf
node's comment prefix and immediately mark the node as formatting-on,
even if a subsequent `# fmt: off` in the same comment prefix would turn
it back off. This change modifies that logic to track the state through
the entire prefix and take the final state.

Note that this does not fully solve on/off behavior, since any _comment_
lines between the off/on are still formatted. We may need to add
virtual leaf nodes to truly solve that. I will leave that for a separate
commit/PR.

Fixes #1005
2019-10-28 20:51:45 +01:00
Łukasz Langa
f99fad1b78
Always move the prefix out when wrapping with parentheses (#1103)
Fixes #1097
2019-10-28 20:34:37 +01:00
Jelle Zijlstra
e027fc9e75 line_length=1 to reduce churn (#1092) 2019-10-28 15:25:42 +01:00
Łukasz Langa
6dca5278a3
Fix regression: unexpected parentheses around non-mathematical powers
This was caused by an overly liberal application of parentheses in #909 that
fixed #646.

Fixes #1041
2019-10-28 14:55:24 +01:00
Joe Antonakakis
df6e1a41f7 Add diff support to blackd (#969) 2019-10-28 14:25:26 +01:00
Jelle Zijlstra
53808e3902 fix crash with long type annotations (#1093) 2019-10-27 11:31:10 +00:00
jgirardet
e9d4e7b67f add gitignore support using pathspec (#878) 2019-10-21 11:44:53 +02:00
Jelle Zijlstra
14b28c89c2
Back out #850 (#1079)
Fixes #1042 (and probably #1044 which looks like the same thing).

The issue with the "obviously unnecessary" parentheses that #850 removed is that sometimes they're necessary to help Black fit something in one line. I didn't see an obvious solution that still removes the parens #850 was intended to remove, so let's back out this change for now in the interest of unblocking a release.

This PR also adds a test adapted from the failing example in #1042, so that if we try to reapply the #850 change we don't break the same case again.
2019-10-20 09:02:17 -07:00
Jelle Zijlstra
a73d25883a
fix CI (#1078) 2019-10-20 08:35:57 -07:00
Augie Fackler
9854d4b375 Tweak collection literals to explode with trailing comma (#826) 2019-10-20 16:08:34 +02:00
Michael J. Sullivan
0ff718e1e2 Blacken .py files in blib2to3 (#1011)
* Blacken .py files in blib2to3

This is in preparation for adding type annotations to blib2to3 in
order to compiling it with mypyc (#1009, which I can rebase on top of
this).

To enforce that it stays blackened, I just cargo-culted the existing
test code used for validating formatting. It feels pretty clunky now,
though, so I can abstract the common logic out into a helper if that
seems better. (But error messages might be less clear then?)

* Tidy up the tests
2019-10-20 15:55:31 +02:00
Michael J. Sullivan
7b11f04d54 Fix type: ignore line breaking when there is a destructuring assignment (#1065)
It turns out we also need to handle invisible *left* parens added at
the *start* of a line. Refactor `contains_unsplittable_type_ignore` to
handle this more cleanly.
2019-10-14 18:15:18 -07:00
Linus Groh
73bd7038fb Add black version header to blackd responses (#1046) 2019-10-13 11:35:31 -07:00
Andrey
6aef6c9d45 #455 Fix bug with tricky unicode symbols (#1047)
* add test for special unicode symbol which usual re can not process correctly
add regex lib which supports unicode 12.1.0 standard
replace re usage in project in favor to regex

* #455 fix dependency
2019-10-13 10:21:15 -07:00
Michael J. Sullivan
1336094634 Fix missed cases in the # type: ignore logic (#1059)
In #1040 I had convinced myself that the type ignore logic didn't
need anything like the ignored_ids from the type comment logic, but I
was wrong, and we do.

We hit these cases in practice a bunch.
2019-10-10 18:25:33 -07:00
Michael J. Sullivan
6fe800933d Fix issue with type comments on lines with trailing commas (#1058)
The code introduced in #1027 to detect whether a type comment appeared
after a regular comment in a Line would spuriously misfire when a leaf
was in the comments dict but had an empty list of comments. This can
occur as an artifact of how comments on trailing commas are handled,
it seems.

(This was discovered trying to test black out on mypy.)
2019-10-10 11:50:51 -07:00
Michael J. Sullivan
d9e71a75cc Don't break long lines when type: ignore is present (#1040)
Fixes #997.
2019-10-02 18:57:49 -07:00
Zsolt Dollenstein
7ae084aaa1 fix tests 2019-09-18 13:33:14 +01:00
Pablo Galindo
673327449f Support PEP 572 in while statements (#1028)
Commit d8fa8df052 added support for
parsing and formatting PEP572, but it missed the posibility to add
PEP572 syntax in while statements.
2019-09-18 12:54:40 +01:00
Michael J. Sullivan
0c44220e21 Don't allow type comments to be merged behind regular comments (#1027)
Type comments only apply if they are the first comment on the line,
which means that allowing them to be pushed behind a regular comment
when joining lines is a semantic change (and, indeed, one that black
catches and fails on).
2019-09-17 13:40:03 -07:00
vezeli
c4d2db4323 Change variable in README according to the PEP8 (#1002)
* Change variable in README according to the PEP8
* Change variable in tests according to the PEP8
2019-09-04 09:11:07 +01:00
Joe Antonakakis
3dc461a41a Fix unstable formatting involving unwrapping multiple parentheses (#836) (#961)
This change also unwraps all unnecessary parentheses.
2019-08-23 15:51:29 +01:00
Zsolt Dollenstein
40e8b3a231 [blackd] Support py36-style values in X-Python-Variant header (#979) 2019-08-13 08:26:01 -07:00
Zsolt Dollenstein
bd0ab3cba2
Reraise exception in skip_if_exception decorator 2019-08-05 10:07:43 +01:00
Joe Antonakakis
154b98579d Fix async blackd tests which won't fail currently (#966) 2019-08-05 10:06:12 +01:00
Joe Antonakakis
c7495b9aa0 Fix unstable format involving backslash + whitespace at beginning of file (#948) 2019-08-04 10:03:19 +01:00
Zsolt Dollenstein
e66451761f appease flake8... 2019-07-28 16:40:04 +01:00
Zsolt Dollenstein
6b5eb7d465 skip tests touching aiohttp when known exception occurs 2019-07-28 16:35:10 +01:00
Zsolt Dollenstein
2848e2e1d6
Support PEP-570 (positional only arguments) (#946)
Code using positional only arguments is considered >= 3.8
2019-07-28 16:17:33 +01:00
Zsolt Dollenstein
d8fa8df052
Add support for walrus operator (#935)
* Parse `:=` properly
* never unwrap parenthesis around `:=`
* When checking for AST-equivalence, use `ast` instead of `typed-ast` when running on python >=3.8
* Assume code that uses `:=` is at least 3.8
2019-07-28 16:03:23 +01:00
Brandt Bucher
b073c9a4e9 Force parentheses between unary op and binary power. (#909) 2019-06-29 09:35:16 -07:00
Zsolt Dollenstein
8f380911e0
Pin comment to single leaf in invisible parens (#872) 2019-06-15 07:29:09 +01:00
dylanjblack
9394de150e Fix trailing comma for function with one arg (#880) (#891)
Modified maybe_remove_trailing_comma to remove trailing commas for
typedarglists (in addition to arglists), and updated line split logic
to ensure that all lines in a function definition that contain only one
arg have a trailing comma.
2019-06-14 21:49:49 -07:00
Zsolt Dollenstein
519c06a8cc
Don't introduce quotes to f-string sub-expressions on string boundaries (#871) 2019-05-26 11:58:00 +02:00
Jelle Zijlstra
957ba24bb6
remove obviously unnecessary parentheses (#850)
Fixes #548
2019-05-15 21:11:04 -07:00
Benjamin Woodruff
448885b256
Move tokenizer config onto grammar, rename flag
Based on the feedback in
https://github.com/python/black/pull/845#issuecomment-490622711

- Remove TokenizerConfig, and add a field to Grammar instead.
- Pass the Grammar to the tokenizer.
- Rename `ASYNC_IS_RESERVED_KEYWORD` to `ASYNC_KEYWORDS` and
  `ASYNC_IS_VALID_IDENTIFIER` to `ASYNC_IDENTIFIERS`.
2019-05-09 17:59:29 +02:00
Benjamin Woodruff
f8617f975d
Add support for always tokenizing async/await as keywords
Fixes #593

I looked into this bug with @ambv and @carljm, and we reached the
conclusion was that it's not possible for the tokenizer to determine if
async/await is a keyword inside all possible generators without breaking
the grammar for older versions of Python.

Instead, we introduce a new tokenizer mode for Python 3.7+ that will
cause all async/await instances to get parsed as a reserved keyword,
which should fix async/await inside generators.
2019-05-09 17:59:29 +02:00
Jason Fried
866be06646 Make --safe work for Python2.7 syntax, by using typed_ast for safe validation (#840) 2019-05-08 07:45:59 -07:00
Carl Meyer
6fdbdb4ee3
Avoid unstable formatting when comment follows escaped newline. (#839). Fixes #767. 2019-05-08 09:53:20 -04:00
Jelle Zijlstra
de806405d2 Add parentheses around tuple unpack assignment (#832)
Fixes #656
2019-05-07 09:51:28 -04:00