To run the formatter on Jupyter Notebooks, Black must be installed
with an extra dependency (`black[jupyter]`). This commit adds an
optional argument to install Black with this dependency when using the
official GitHub Action [1]. To enable the formatter on Jupyter
Notebooks, just add `jupyter: true` as an argument. Feature requested
at [2].
[1]: https://black.readthedocs.io/en/stable/integrations/github_actions.html
[2]: https://github.com/psf/black/issues/3280
Signed-off-by: Antonio Ossa Guerra <aaossa@uc.cl>
This implements PEP 621, obviating the need for `setup.py`, `setup.cfg`,
and `MANIFEST.in`. The build backend Hatchling (of which I am a
maintainer in the PyPA) is now used as that is the default in the
official Python packaging tutorial. Hatchling is available on all the
major distribution channels such as Debian, Fedora, and many more.
## Python support
The earliest supported Python 3 version of Hatchling is 3.7, therefore
I've also set that as the minimum here. Python 3.6 is EOL and other
build backends like flit-core and setuptools also dropped support.
Python 3.6 accounted for 3-4% of downloads in the last month.
## Plugins
Configuration is now completely static with the help of 3 plugins:
### Readme
hynek's hatch-fancy-pypi-readme allows for the dynamic construction of
the readme which was previously coded up in `setup.py`. Now it's simply:
```toml
[tool.hatch.metadata.hooks.fancy-pypi-readme]
content-type = "text/markdown"
fragments = [
{ path = "README.md" },
{ path = "CHANGES.md" },
]
```
### Versioning
hatch-vcs is currently just a wrapper around setuptools-scm (which
despite the legacy naming is actually now decoupled from setuptools):
```toml
[tool.hatch.version]
source = "vcs"
[tool.hatch.build.hooks.vcs]
version-file = "src/_black_version.py"
template = '''
version = "{version}"
'''
```
### mypyc
hatch-mypyc offers many benefits over the existing approach:
- No need to manually select files for inclusion
- Avoids the need for the current CI workaround for https://github.com/mypyc/mypyc/issues/946
- Intermediate artifacts (like `build/`) from setuptools and mypyc
itself no longer clutter the project directory
- Runtime dependencies required at build time no longer need to be
manually redeclared as this is a built-in option of Hatchling
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Previously _Black_ produces invalid code because the `# fmt: on` is used on a different block level.
While _Black_ requires `# fmt: off` and `# fmt: on` to be used at the same block level, incorrect usage shouldn't cause crashes.
The formatting behavior this PR introduces is, the code below the initial `# fmt: off` block level will be turned off for formatting, when `# fmt: on` is used on a different level or there is no `# fmt: on`. This also matches the current behavior when `# fmt: off` is used at the top-level without a matching `# fmt: on`, it turns off formatting for everything below `# fmt: off`.
- Fixes#2567
- Fixes#3184
- Fixes#2985
- Fixes#2882
- Fixes#2232
- Fixes#2140
- Fixes#1817
- Fixes#569
Bumps cibuildwheel from 2.8.1 to 2.10.0 which has 3.11 building enabled
by default. Unfortunately mypyc errors out on 3.11:
src/black/files.py:29:9: error: Name "tomllib" already defined (by an import) [no-redef]
... so we have to also hide the fallback import of tomli on older 3.11
alphas from mypy[c].
Make sure `gcc` is installed in the build env
The mypyc build requires `gcc` to be installed even if it's being built with `clang`, otherwise `clang` fails to find `libgcc`.
These two paragraphs were tucked away at the end of the section, after
the diversion on backslashes. I nearly missed the first paragraph and
opened a nonsense issue, and I think the second belongs higher up with
it too.
Fix a crash when formatting some dicts with parenthesis-wrapped long
string keys. When LL[0] is an atom string, we need to check the atom
node's siblings instead of LL[0] itself, e.g.:
dictsetmaker
atom
STRING '"This is a really long string that can\'t be expected to fit in one line and is used as a nested dict\'s key"'
/atom
COLON ':'
atom
LSQB ' ' '['
listmaker
STRING '"value"'
COMMA ','
STRING ' ' '"value"'
/listmaker
RSQB ']'
/atom
COMMA ','
/dictsetmaker
* Move 311 tests to install aiohttp without C extensions
- Configure tox to install aiohttp without extensions
- i.e. use `AIOHTTP_NO_EXTENSIONS=1` for pip install
- This allows us to reenable blackd tests that use aiohttp testing helpers etc.
- Had to ignore `cgi` module deprecation warning
- Filed issue for aiohttp to fix: https://github.com/aio-libs/aiohttp/issues/6905
Test:
- `/tmp/tb/bin/tox -e 311`
* Fix formatting + linting
* Add latest aiohttp for loop fix + Try to exempt deprecation warning but failed - will ask for help
* Remove unnecessary warning ignore
Co-authored-by: Cooper Ry Lees <me@wcooperlees.com>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
This is deprecated since aiohttp 4.0. If it doesn't exist just define a
no-op decorator that does nothing (after the other aiohttp imports
though!). By doing this, it's safe to ignore the DeprecationWarning
without needing to require the latest aiohttp once they remove
`@middleware`.
We've decided to a) convert stable back into a branch and b) to update
it immediately as part of the release process. We may as well automate
it. And about going back to a branch ...
Git tags are not the right tool, at all[^1]. They come with the
expectation that they will never change. Things will not work as
expected if they do change, doubly so if they change regularly. Once
you pull stable from the remote and it's copied in your local
repository, no matter how many times you run git pull you'll never see
it get updated automatically. Your only recourse is to delete the tag
via `git tag -d stable` before pulling.
This gets annoying really quickly since stable is supposed to be the
solution for folks "who want to move along as Black developers deem
the newest version reliable."[^2] See this comment for how this impacts
users using our Vim plugin[^3]. It also affects us developers[^4]. If
you have stable locally, once we cut a new release and update the stable
tag, a simple `git pull` / `git fetch` will not pull down the updated
stable tag. Unless you remember to delete stable before pulling, stable
will become stale and useless.
You can argue this is a good thing ("people should explicitly opt into
updating stable"), but IMO it does not match user expectations nor
developer expectations[^5]. Especially since not all our integrations
that use stable are bound by this security measure, for example our
GitHub Action (since it does a clean fetch of the repository every time
it's used). I believe consistency would be good here.
Finally, ever since we switched to a tag, we've been facing issues with
ReadTheDocs not picking up updates to stable unless we force a rebuild.
The initial rebuild on the stable update just pulls the commit the tag
previously pointed to. I'm not sure if switching back to a branch will
fix this, but I'd wager it will.
[^1]: https://git-scm.com/docs/git-tag#_on_re_tagging
[^2]: https://black.readthedocs.io/en/stable/contributing/release_process.html#moving-the-stable-tag
[^3]: https://github.com/psf/black/issues/2503#issuecomment-1196357379
[^4]: In fairness, most folks working on Black probably don't use the
`stable` ref anyway, especially us maintainers who'd know what is
the latest version by heart, but it'd still be nice to make it
usable for local dev though.
[^5]: Also what benefit does a `stable` ref have over explicit version
tags like `22.6.0`? If you're going to opt into some odd pin
mechanism, might as well use explicit version tags for clarity
and consistency.
- Formalise release cadence guidelines
- Overhaul release steps to be easier to follow and more thorough
- Reorder changelog template to something more sensible
- Update release automation docs to reflect recent improvements (notably
the addition of in-repo mypyc wheel builds)
Co-authored-by: Felix Hildén <felix.hilden@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Solves https://github.com/psf/black/issues/2598 where Black wouldn't
use .gitignore at folder/.gitignore if you ran `black folder` for
example.
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Adds parentheses around implicit string concatenations when it's inside
a list, set, or tuple. Except when it's only element and there's no trailing
comma.
Looking at the order of the transformers here, we need to "wrap in
parens" before string_split runs. So my solution is to introduce a
"collaboration" between StringSplitter and StringParenWrapper where the
splitter "skips" the split until the wrapper adds the parens (and then
the line after the paren is split by StringSplitter) in another pass.
I have also considered an alternative approach, where I tried to add a
different "string paren wrapper" class, and it runs before string_split.
Then I found out it requires a different do_transform implementation
than StringParenWrapper.do_transform, since the later assumes it runs
after the delimiter_split transform. So I stopped researching that
route.
Originally function calls were also included in this change, but given
missing commas should usually result in a runtime error and the scary
amount of changes this cause on downstream code, they were removed in
later revisions.
os.cpu_count() can return None (sounds like a super arcane edge case
though) so the type annotation for the `workers` parameter of
`black.main` is wrong. This *could* technically cause a runtime
TypeError since it'd trip one of mypyc's runtime type checks so we
might as well fix it.
Reading the documentation (and cross-checking with the source code),
you are actually allowed to pass None as `max_workers` to
`concurrent.futures.ProcessPoolExecutor`. If it is None, the pool
initializer will simply call os.cpu_count() [^1] (defaulting to 1 if it
returns None [^2]). It'll even round down the worker count to a level
that's safe for Windows.
... so theoretically we don't even need to call os.cpu_count()
ourselves, but our Windows limit is 60 (unlike the stdlib's 61) and I'd
prefer not accidentally reintroducing a crash on machines with many,
many CPU cores.
[^1]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor
[^2]: a372a7d653/Lib/concurrent/futures/process.py (L600)
Loading .gitignore and compiling the exclude regex can take more than
15ms. We shouldn't and don't need to pay this cost if we're simply
formatting files given on the command line directly.
I would've loved to lazily import pathspec, but the patch won't be clean
until the file collection and discovery logic is refactored first.
Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com>
`black.reformat_many` depends on a lot of slow-to-import modules. When
formatting simply a single file, the time paid to import those modules
is totally wasted. So I moved `black.reformat_many` and its helpers
to `black.concurrency` which is now *only* imported if there's more
than one file to reformat. This way, running Black over a single file
is snappier
Here are the numbers before and after this patch running `python -m
black --version`:
- interpreted: 411 ms +- 9 ms -> 342 ms +- 7 ms: 1.20x faster
- compiled: 365 ms +- 15 ms -> 304 ms +- 7 ms: 1.20x faster
Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com>
There are a number of places this behaviour could be patched, for
instance, it's quite tempting to patch it in `get_sources`. However
I believe we generally have the invariant that project root contains all
files we want to format, in which case it seems prudent to keep that
invariant.
This also improves the accuracy of the "sources to be formatted" log
message with --stdin-filename.
Fixes GH-3207.
Updates action.yml to use the alternative $GITHUB_ACTION_PATH variable
instead of the original ${{ github.action_path }} which caused issues
with bash on the Windows runners. This removes the need for a Python
subprocess to call the main.py script.
Fixes#2734: a standalone comment causes strings to be merged into one far too long (and requiring two passes to do so).
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
- Had to exempt blackd tests for now due to aiohttp
- Skip by using `sys.version_info` tuple
- aiohttp does not compile in 3.11 yet - refer to #3230
- Add a deadsnakes ubuntu workflow to run 3.11-dev to ensure we don't regress
- Have it also format ourselves
Test:
- `tox -e 311`
Co-authored-by: Cooper Ry Lees <me@wcooperlees.com>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
uR is not a legal string prefix, so this test breaks (AssertionError:
cannot use --safe with this file; failed to parse source file AST:
invalid syntax) if changed to one in which the file is changed. I've
changed the last test to have u alone, and added an R to the test above
instead.
* makes install available for all users in docker image
moves the installation path from /root/.local to a
virtualenv. this way we still get the lightweight
multistage build without excluding non-root users.
* adds changelog entry for docker-image fix
A changelog entry has been added under the Integration
subheader
* changes dockerfile to use the venv activate script
we are now using the inbuilt venv activate script, as well
as explicitly mentioning the binary location in the entrypoint
cmd.
Co-authored-by: Nicolò <nicolo.intrieri@spinforward.it>
Co-authored-by: Cooper Lees <me@cooperlees.com>
Building executables without any testing is quite sketchy, let's at
least verify they won't crash on startup and format Black's own
codebase.
Also replaced "binaries" with "executables" since it's clearer and
won't be confused with mypyc.
Finally, I added colorama so all Windows users can get colour.
As error logs are emitted often (they happen when Black's cache
directory is created after blib2to3 tries to write its cache) and cause
issues to be filed by users who think Black isn't working correctly.
These errors are expected for now and aren't a cause for concern so
let's remove them to stop worrying users (and new issues from being
opened). We can improve the blib2to3 caching mechanism to write its
cache at the end of a successful command line invocation later.
As mentioned in GH-3185, when using Black as a Vim plugin, especially
automatically on save, the plugin's messages can be confusing, as
nothing indicates that they come from Black.
