We've decided to a) convert stable back into a branch and b) to update
it immediately as part of the release process. We may as well automate
it. And about going back to a branch ...
Git tags are not the right tool, at all[^1]. They come with the
expectation that they will never change. Things will not work as
expected if they do change, doubly so if they change regularly. Once
you pull stable from the remote and it's copied in your local
repository, no matter how many times you run git pull you'll never see
it get updated automatically. Your only recourse is to delete the tag
via `git tag -d stable` before pulling.
This gets annoying really quickly since stable is supposed to be the
solution for folks "who want to move along as Black developers deem
the newest version reliable."[^2] See this comment for how this impacts
users using our Vim plugin[^3]. It also affects us developers[^4]. If
you have stable locally, once we cut a new release and update the stable
tag, a simple `git pull` / `git fetch` will not pull down the updated
stable tag. Unless you remember to delete stable before pulling, stable
will become stale and useless.
You can argue this is a good thing ("people should explicitly opt into
updating stable"), but IMO it does not match user expectations nor
developer expectations[^5]. Especially since not all our integrations
that use stable are bound by this security measure, for example our
GitHub Action (since it does a clean fetch of the repository every time
it's used). I believe consistency would be good here.
Finally, ever since we switched to a tag, we've been facing issues with
ReadTheDocs not picking up updates to stable unless we force a rebuild.
The initial rebuild on the stable update just pulls the commit the tag
previously pointed to. I'm not sure if switching back to a branch will
fix this, but I'd wager it will.
[^1]: https://git-scm.com/docs/git-tag#_on_re_tagging
[^2]: https://black.readthedocs.io/en/stable/contributing/release_process.html#moving-the-stable-tag
[^3]: https://github.com/psf/black/issues/2503#issuecomment-1196357379
[^4]: In fairness, most folks working on Black probably don't use the
`stable` ref anyway, especially us maintainers who'd know what is
the latest version by heart, but it'd still be nice to make it
usable for local dev though.
[^5]: Also what benefit does a `stable` ref have over explicit version
tags like `22.6.0`? If you're going to opt into some odd pin
mechanism, might as well use explicit version tags for clarity
and consistency.
- Formalise release cadence guidelines
- Overhaul release steps to be easier to follow and more thorough
- Reorder changelog template to something more sensible
- Update release automation docs to reflect recent improvements (notably
the addition of in-repo mypyc wheel builds)
Co-authored-by: Felix Hildén <felix.hilden@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Solves https://github.com/psf/black/issues/2598 where Black wouldn't
use .gitignore at folder/.gitignore if you ran `black folder` for
example.
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Adds parentheses around implicit string concatenations when it's inside
a list, set, or tuple. Except when it's only element and there's no trailing
comma.
Looking at the order of the transformers here, we need to "wrap in
parens" before string_split runs. So my solution is to introduce a
"collaboration" between StringSplitter and StringParenWrapper where the
splitter "skips" the split until the wrapper adds the parens (and then
the line after the paren is split by StringSplitter) in another pass.
I have also considered an alternative approach, where I tried to add a
different "string paren wrapper" class, and it runs before string_split.
Then I found out it requires a different do_transform implementation
than StringParenWrapper.do_transform, since the later assumes it runs
after the delimiter_split transform. So I stopped researching that
route.
Originally function calls were also included in this change, but given
missing commas should usually result in a runtime error and the scary
amount of changes this cause on downstream code, they were removed in
later revisions.
os.cpu_count() can return None (sounds like a super arcane edge case
though) so the type annotation for the `workers` parameter of
`black.main` is wrong. This *could* technically cause a runtime
TypeError since it'd trip one of mypyc's runtime type checks so we
might as well fix it.
Reading the documentation (and cross-checking with the source code),
you are actually allowed to pass None as `max_workers` to
`concurrent.futures.ProcessPoolExecutor`. If it is None, the pool
initializer will simply call os.cpu_count() [^1] (defaulting to 1 if it
returns None [^2]). It'll even round down the worker count to a level
that's safe for Windows.
... so theoretically we don't even need to call os.cpu_count()
ourselves, but our Windows limit is 60 (unlike the stdlib's 61) and I'd
prefer not accidentally reintroducing a crash on machines with many,
many CPU cores.
[^1]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor
[^2]: a372a7d653/Lib/concurrent/futures/process.py (L600)
Loading .gitignore and compiling the exclude regex can take more than
15ms. We shouldn't and don't need to pay this cost if we're simply
formatting files given on the command line directly.
I would've loved to lazily import pathspec, but the patch won't be clean
until the file collection and discovery logic is refactored first.
Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com>
`black.reformat_many` depends on a lot of slow-to-import modules. When
formatting simply a single file, the time paid to import those modules
is totally wasted. So I moved `black.reformat_many` and its helpers
to `black.concurrency` which is now *only* imported if there's more
than one file to reformat. This way, running Black over a single file
is snappier
Here are the numbers before and after this patch running `python -m
black --version`:
- interpreted: 411 ms +- 9 ms -> 342 ms +- 7 ms: 1.20x faster
- compiled: 365 ms +- 15 ms -> 304 ms +- 7 ms: 1.20x faster
Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com>
There are a number of places this behaviour could be patched, for
instance, it's quite tempting to patch it in `get_sources`. However
I believe we generally have the invariant that project root contains all
files we want to format, in which case it seems prudent to keep that
invariant.
This also improves the accuracy of the "sources to be formatted" log
message with --stdin-filename.
Fixes GH-3207.
Updates action.yml to use the alternative $GITHUB_ACTION_PATH variable
instead of the original ${{ github.action_path }} which caused issues
with bash on the Windows runners. This removes the need for a Python
subprocess to call the main.py script.
Fixes#2734: a standalone comment causes strings to be merged into one far too long (and requiring two passes to do so).
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
- Had to exempt blackd tests for now due to aiohttp
- Skip by using `sys.version_info` tuple
- aiohttp does not compile in 3.11 yet - refer to #3230
- Add a deadsnakes ubuntu workflow to run 3.11-dev to ensure we don't regress
- Have it also format ourselves
Test:
- `tox -e 311`
Co-authored-by: Cooper Ry Lees <me@wcooperlees.com>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
uR is not a legal string prefix, so this test breaks (AssertionError:
cannot use --safe with this file; failed to parse source file AST:
invalid syntax) if changed to one in which the file is changed. I've
changed the last test to have u alone, and added an R to the test above
instead.
* makes install available for all users in docker image
moves the installation path from /root/.local to a
virtualenv. this way we still get the lightweight
multistage build without excluding non-root users.
* adds changelog entry for docker-image fix
A changelog entry has been added under the Integration
subheader
* changes dockerfile to use the venv activate script
we are now using the inbuilt venv activate script, as well
as explicitly mentioning the binary location in the entrypoint
cmd.
Co-authored-by: Nicolò <nicolo.intrieri@spinforward.it>
Co-authored-by: Cooper Lees <me@cooperlees.com>
Building executables without any testing is quite sketchy, let's at
least verify they won't crash on startup and format Black's own
codebase.
Also replaced "binaries" with "executables" since it's clearer and
won't be confused with mypyc.
Finally, I added colorama so all Windows users can get colour.
As error logs are emitted often (they happen when Black's cache
directory is created after blib2to3 tries to write its cache) and cause
issues to be filed by users who think Black isn't working correctly.
These errors are expected for now and aren't a cause for concern so
let's remove them to stop worrying users (and new issues from being
opened). We can improve the blib2to3 caching mechanism to write its
cache at the end of a successful command line invocation later.
As mentioned in GH-3185, when using Black as a Vim plugin, especially
automatically on save, the plugin's messages can be confusing, as
nothing indicates that they come from Black.
... in the middle of an expression or code block by adding a missing return.
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
When the Leaf node with `# fmt: skip` is a NEWLINE inside a `suite`
Node, the nodes to ignore should be from the siblings of the parent
`suite` Node.
There is a also a special case for the ASYNC token, where it expands
to the grandparent Node where the ASYNC token is.
This fixes GH-2646, GH-3126, GH-2680, GH-2421, GH-2339, and GH-2138.
The former was a regression I introduced a long time ago. To avoid
changing the stable style too much, the regression is only fixed if
--preview is enabled
Annoyingly enough, as we currently always enforce a second format pass if
changes were made, there's no good way to prove the existence of the
docstring quote normalization instability issue. For posterity, here's
one failing example:
--- source
+++ first pass
@@ -1,7 +1,7 @@
def some_function(self):
- ''''<text here>
+ """ '<text here>
<text here, since without another non-empty line black is stable>
- '''
+ """
pass
--- first pass
+++ second pass
@@ -1,7 +1,7 @@
def some_function(self):
- """ '<text here>
+ """'<text here>
<text here, since without another non-empty line black is stable>
"""
pass
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* Move to explicitly creating a new loop
- >= 3.10 add a warning that `get_event_loop` will not automatically create a loop
- Move to explicit API
Test:
- `python3.11 -m venv --upgrade-deps /tmp/tb`
- `/tmp/tb/bin/pip install -e .`
- Install deps and no blackd as aiohttp + yarl can't build still with 3.11
- https://github.com/aio-libs/aiohttp/issues/6600
- `export PYTHONWARNINGS=error`
```
cooper@l33t:~/repos/black$ /tmp/tb/bin/black .
All done! ✨🍰✨
44 files left unchanged.
```
Fixes#3110
* Add to CHANGES.md
* Fix a cooper typo yet again
* Set default asyncio loop to our explicitly created one + unset on exit
* Update CHANGES.md
Fix my silly typo.
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Cooper Ry Lees <me@wcooperlees.com>
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Doing so is invalid. Note this only fixes the preview style since the
logic putting closing docstring quotes on their own line if they violate
the line length limit is quite new.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Otherwise they'd be deleted which was a regression in 22.1.0 (oops! my
bad!). Also type comments are now tracked in the AST safety check on all
compatible platforms to error out if this happens again.
Overall the line rewriting code has been rewritten to do "the right
thing (tm)", I hope this fixes other potential bugs in the code (fwiw I
got to drop the bugfix in blib2to3.pytree.Leaf.clone since now bracket
metadata is properly copied over).
Fixes#2873
* Recommend using BlackConnect in IntelliJ IDEs
* IntelliJ IDEs integration docs: improve formatting
* Add changelog for recommending BlackConnect
* IntelliJ IDEs integration docs: improve formatting
* Apply suggestions from code review
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
* Fix indentation
* Apply italic to Black name
Consequently with other places in the document
* Move CHANGELOG entry to Unreleased section
* IntelliJ IDEs integration docs: bring back a point with formatting a file
* IntelliJ IDEs integration docs: fix extra whitespace and linebreak
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Covers GH-2926, GH-2990, GH-2991, and GH-3035.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
