Closes#2164.
Changes behavior of how .gitignore is handled. With this change, the rules in .gitignore are only used as a fallback if no exclusion rule is explicitly passed on the command line or in pyproject.toml. Previously they were used regardless if explicit exclusion rules were specified, preventing any overriding of .gitignore rules.
Those that depend only on .gitignore for their exclusion rules will not be affected. Those that use both .gitignore and exclude will find that exclude will act more like actually specifying exclude and not just another extra-excludes. If the previous behavior was desired, they should move their rules from exclude to extra-excludes.
We're only fixing them so fuzzers don't yell at us when we break "valid"
code. I mean "valid" because some of the examples aren't even accepted by
Python.
Optional trailing commas put by Black become magic trailing commas on another
pass of the tool. Since they are influencing formatting around optional
parentheses, on rare occasions the tool changes its mind in terms of putting
parentheses or not.
Ideally this would never be the case but sadly the decision to put optional
parentheses or not (which looks at pre-existing "magic" trailing commas) is
happening around the same time as the decision to put an optional trailing
comma. Untangling the two proved to be impractically difficult.
This shameful workaround uses the fact that the formatting instability
introduced by magic trailing commas is deterministic: if the optional trailing
comma becoming a pre-existing "magic" trailing comma changes formatting, the
second pass becomes stable since there is no variable factor anymore on pass 3,
4, and so on.
For most files, this will introduce no performance penalty since `--safe` is
already re-formatting everything twice to ensure formatting stability. We're
using this result and if all's good, the behavior is equivalent. If there is
a difference, we treat the second result as the binding one, and check its
sanity again.
* Added support for top-level user configuration
At the user level, a TOML config can be specified in the following locations:
* Windows: ~\.black
* Unix-like: $XDG_CONFIG_HOME/black (~/.config/black fallback)
Instead of changing env vars for the entire black-primer process, they
are now changed only for the black subprocess, using a tmpdir.
Fixes: #1662
Work-around for https://bugs.python.org/issue2142
The test has to slightly mess with its input data, because the utility
functions default to ensuring the test data has a final newline, which
defeats the point of the test.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
* Add test case to illustrate the issue
* Accept carriage returns as valid separators while enumerating comments
Without this acceptance, escaped multi-line statments that use carriage returns will not be counted into the 'ignored_lines' variable since the emitted line values will end with a CR and not an escape character. That leads to comments associated with the line being incorrectly labeled with the STANDALONE_COMMENT type, affecting comment placement and line space management.
* Remove comment linking to ephemeral build log
* Provide a stdin-filename to allow stdin to respect exclude/force-exclude rules
This will allow automatic tools to enforce the project's
exclude/force-exclude rules even if they pass the file through stdin to
update its buffer.
This is a similar solution to --stdin-display-name in flake8.
* Update src/black/__init__.py
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
* --stdin-filename should only respect --exclude-filename
* Update README with the new --stdin-filename option
* Write some tests for the new stdin-filename functionality
* Apply suggestions from code review
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Force stdin output when we asked for stdin even if the file exists
* Add an entry in the changelog regarding --stdin-filename
* Reduce disk reads if possible
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
* Check for is_stdin and p.is_file before checking for p.is_dir()
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Black used to erroneously remove all empty lines between non-function
code and decorators when formatting typing stubs. Now a single empty
line is enforced.
I chose for putting empty lines around decorated classes that have empty
bodies since removing empty lines around such classes would cause a
formatting issue that seems to be impossible to fix.
For example:
```
class A: ...
@some_decorator
class B: ...
class C: ...
class D: ...
@some_other_decorator
def foo(): -> None: ...
```
It is easy to enforce no empty lines between class A, B, and C.
Just return 0, 0 for a line that is a decorator and precedes an stub
class. Fortunately before this commit, empty lines after that class
would be removed already.
Now let's look at the empty line between class D and function foo. In
this case, there should be an empty line there since it's class code next
to function code. The problem is that when deciding to add X empty lines
before a decorator, you can't tell whether it's before a class or a
function. If the decorator is before a function, then an empty line
is needed, while no empty lines are needed when the decorator is
before a class.
So even though I personally prefer no empty lines around decorated
classes, I had to go the other way surrounding decorated classes with
empty lines.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
The quotes of multiline docstrings are now only normalized when string
normalization is off, instead of the string normalization setting being
ignored and the quotes being *always* normalized.
I had to make a new test case and data file since the current pair for
docstrings only worked when there is no formatting difference between the
formatting results with string normalization on and off. I needed to add
tests for when there *are* differences between the two. So I split
test_docstring's test code when string normalization is disabled into a
new test case along with a new data file.
This addresses a few crashers, namely:
* producing non-equivalent code due to mangling escaped newlines,
* invalid hugging quote characters in the docstring body to the docstring outer
triple quotes (causing a quadruple quote which is a syntax error),
* lack of handling for docstrings that start on the same line as the `def`, and
* invalid stripping of outer triple quotes when the docstring contained
a string prefix.
As a bonus, tests now also run when string normalization is disabled.
This required some hackery. Long story short, we need to reuse the ability to
omit rightmost bracket pairs (which glues them together and splits on something
else instead), for use with pre-existing trailing commas.
This form of user-controlled formatting is brittle so we have to be careful not
to cause a scenario where Black first formats code without trailing commas in
one way, and then looks at the same file with pre-existing trailing commas
(that it itself put on the previous run) and decides to format the code again.
One particular ugly edge case here is handling of optional parentheses. In
particular, the long-standing `line_length=1` hack got in the way of
pre-existing trailing commas and had to be removed. Instead, a more
intelligent but costly solution was put in place: a "second opinion" if the
formatting that omits optional parentheses ended up causing lines to be too
long. Again, for efficiency purposes, Black reuses Leaf objects from blib2to3
and modifies them in place, which was invalid for having two separate
formattings. Line cloning was used to mitigate this.
Fixes#1619
- when a trailing comma is specified in any bracket pair, that signals to Black
that this bracket pair needs to be always exploded, e.g. presented as "one
item per line";
- this causes some changes to previously formatted code that erroneously left
trailing commas embedded into single-line expressions;
- internally, Black needs to be able to identify trailing commas that it put
itself compared to pre-existing trailing commas. We do this by using/abusing
lib2to3's `was_checked` attribute. It's True for internally generated
trailing commas and False for pre-existing ones (in fact, for all
pre-existing leaves and nodes).
Fixes#1288
* put experimental string stuff behind a flag
* update tests
* don't need an output section if it's the same as the input
* Primer: Expect no formatting changes in attrs, hypothesis and poetry with --experimental-string-processing off
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Ever since --force-exclude was added, --exclude started to touch files
that were given to Black through the CLI too. This is not documented
behaviour and neither expected as --exclude and --force-exclude now
behave the same!
Before this commit, get_sources() when encountering a file that was passed
explicitly through the CLI would pass a single Path object list to
gen_python_files(). This causes bad behaviour since that function
doesn't treat the exclude and force_exclude regexes differently. Which
is fine for recursively found files, but *not* for files given through
the CLI.
Now when get_sources() iterates through srcs and encounters
a file, it checks if the force_exclude regex matches, if not, then the
file will be added to the computed sources set.
A new function had to be created since before you can do regex matching,
the path must be normalized. The full process of normalizing the path is
somewhat long as there is special error handling. I didn't want to
duplicate this logic in get_sources() and gen_python_files() so that's
why there is a new helper function.
* Bump toml from 0.10.0 to 0.10.1 to fix a bug
* Add tests for TOML parsing and reading
* Fix configuration bug affecting vim plugin
The vim plugin directly calls parse_pyproject and skips the Click processing
, but parse_pyproject assumed that it would only be used before Click processing
and therefore made the config values click friendly. This moves the "make the values
click friendly processing" into read_pyproject_toml which is only called by a Click
callback.
* Please mypy and flake8
* Add black-primer unittests
- Get this tool covered with some decent unittests for all unittests wins
- Have a CLI and lib test class
- Import it from `test_black.py` so we always run tests
- Revert typing asyncio.Queue as Queue[str] so we can work in 3.6
- **mypy**: Until black > 3.6 disallow_any_generics=False for primer code
Test:
- Run tests: `coverage run tests/test_primer.py` or `coverage run -m unittest`
```
(b) cooper-mbp1:black cooper$ coverage report
Name Stmts Miss Cover
---------------------------------------------
src/black_primer/cli.py 49 8 84%
src/black_primer/lib.py 148 28 81%
tests/test_primer.py 114 1 99%
---------------------------------------------
TOTAL 311 37 88%
```
* Use ProactorEventLoop for Windows + fix false path for Linux
* Set Windows to use ProactorEventLoop in to benefit all callers
* sys.platform seems to not having the loop applied - So type ignore and use platform.system() gate
* Have each test loop correctly set to ProactorEventLoop on Windows for < 3.8 too
- Move black.py to src/black/__init__.py
- Have setuptools_scm make src/_black_version.py and exclude from git
- Move blackd.py to src/blackd/__init__.py
- Move blib2to3/ to src/
- Update `setup.py`
- Update unittests to pass
- Mostly path fixing + resolving
- Update CI
- pre-commit config
- appveyor + travis
Tested on my mac with python 3.7.5 via:
```
python3 -m venv /tmp/tb3
/tmp/tb3/bin/pip install --upgrade setuptools pip coverage pre-commit
/tmp/tb2/bin/pip install ~/repos/black/
cd ~/repos/black/
/tmp/tb2/bin/coverage run tests/test_black.py
/tmp/tb3/bin/pre-commit run -a
/tmp/tb3/bin/black --help
/tmp/tb3/bin/black ~/repos/ptr/ptr.py
```
AWS Lambda and some other virtualized environment may not permit access
to /dev/shm on Linux and as such, trying to use ProcessPoolExecutor will
fail.
As using parallelism is only a 'nice to have' feature of black, if it fails
we gracefully fallback to a monoprocess implementation, which permits black
to finish normally.
Co-authored-by: Allan Simon <asimon@yolaw.fr>
* Re-indent the contents of docstrings when indentation changes
Keeping the contents of docstrings completely unchanged when
re-indenting (from 2-space intents to 4, for example) can cause
incorrect docstring indentation:
```
class MyClass:
"""Multiline
class docstring
"""
def method(self):
"""Multiline
method docstring
"""
pass
```
...becomes:
```
class MyClass:
"""Multiline
class docstring
"""
def method(self):
"""Multiline
method docstring
"""
pass
```
This uses the PEP 257 algorithm for determining docstring indentation,
and adjusts the contents of docstrings to match their new indentation
after `black` is applied.
A small normalization is necessary to `assert_equivalent` because the
trees are technically no longer precisely equivalent -- some constant
strings have changed. When comparing two ASTs, whitespace after
newlines within constant strings is thus folded into a single space.
Co-authored-by: Luka Zakrajšek <luka@bancek.net>
* Extract the inner `_v` method to decrease complexity
This reduces the cyclomatic complexity to a level that makes flake8 happy.
* Blacken blib2to3's docstring which had an over-indent
Co-authored-by: Luka Zakrajšek <luka@bancek.net>
Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>
This pull request's main intention is to wraps long strings (as requested by #182); however, it also provides better string handling in general and, in doing so, closes the following issues:
Closes#26Closes#182Closes#933Closes#1183Closes#1243
Fixes#1360, where an invalid config file causes a return/exit code of 1. This
change means this case is caught earlier, treated like any other bad
parameters, and results in an exit code of 2.
Co-authored-by: Toby Fleming <tobywf@users.noreply.github.com>
