* Add release tool
- Add tool for release managers to use to generate commits
- I'm trying to only use stdlib so we have no depdencies ...
- Default is to change date strings in hard coded documentation files + CHANGES.md
- I write directly to files cause we have SCM to fix any screw ups ...
- We hackily convert calver to ints to sort (all for better ideas here)
- If we hit a ValueError we just set to 0 for sorting - This is alhpa + beta release we can safely ignore these days
- Add new CI to only run release unittests in 3.12 only on all platforms
- Update release docs
- Checked with `mypy --strict` + ensure we are `black --preview` formatted :D
Tests:
- Run it to generate template PR
- `python3.12 release.py --debug --add-changes-template`
- Run it to cleanup CHANGE.md + change version in specified doc files
```
crl-m1:black cooper$ python3.12 release.py -d
[2023-10-23 23:39:38,414] INFO: Current version detected to be 23.10.1 (release.py:221)
[2023-10-23 23:39:38,414] INFO: Next version will be 23.10.2 (release.py:222)
[2023-10-23 23:39:38,414] INFO: Cleaning up /Users/cooper/repos/black/CHANGES.md (release.py:127)
[2023-10-23 23:39:38,416] DEBUG: Finished Cleaning up /Users/cooper/repos/black/CHANGES.md (release.py:147)
[2023-10-23 23:39:38,416] INFO: Updating black version to 23.10.2 in /Users/cooper/repos/black/docs/integrations/source_version_control.md (release.py:173)
[2023-10-23 23:39:38,416] DEBUG: Finished updating black version to 23.10.2 in /Users/cooper/repos/black/docs/integrations/source_version_control.md (release.py:185)
[2023-10-23 23:39:38,416] INFO: Updating black version to 23.10.2 in /Users/cooper/repos/black/docs/usage_and_configuration/the_basics.md (release.py:173)
[2023-10-23 23:39:38,417] DEBUG: Finished updating black version to 23.10.2 in /Users/cooper/repos/black/docs/usage_and_configuration/the_basics.md (release.py:185)
```
- Add tests around some key logic
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lints + add git to release CI
- Remove black + mypy as linting already runs it ...
- Ignore delete param to TemporaryDirectory as we can't set mypy to 3.12 :D
* Only run CI on linux/ubuntu for now
* Add lots of debug printing + directly run unitests (not via coverage)
* Overloading __str__ is bad on a TestCase
* Add more logging around git tag
* Print where git is in a step
* Rollback creating a fake black repo as we were not using it - I did plan to but I can't get it working on GitHub actions
* Do a deep checkout
* Add noqa for E701,E761 ... maybe we need this in our flake8 config now?
* Fix action to have correct workflow yaml to action on
- Also add fix to not double run when we push directly to psf/black
* All jelle suggestions
- Fix bug missing lines ending with --> in CHANGES.md to delete ...
- Update ci to run out of scripts dir too
- Update test_tuple_calver
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Previously _Black_ produces invalid code because the `# fmt: on` is used on a different block level.
While _Black_ requires `# fmt: off` and `# fmt: on` to be used at the same block level, incorrect usage shouldn't cause crashes.
The formatting behavior this PR introduces is, the code below the initial `# fmt: off` block level will be turned off for formatting, when `# fmt: on` is used on a different level or there is no `# fmt: on`. This also matches the current behavior when `# fmt: off` is used at the top-level without a matching `# fmt: on`, it turns off formatting for everything below `# fmt: off`.
- Fixes#2567
- Fixes#3184
- Fixes#2985
- Fixes#2882
- Fixes#2232
- Fixes#2140
- Fixes#1817
- Fixes#569
- Formalise release cadence guidelines
- Overhaul release steps to be easier to follow and more thorough
- Reorder changelog template to something more sensible
- Update release automation docs to reflect recent improvements (notably
the addition of in-repo mypyc wheel builds)
Co-authored-by: Felix Hildén <felix.hilden@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
`black.reformat_many` depends on a lot of slow-to-import modules. When
formatting simply a single file, the time paid to import those modules
is totally wasted. So I moved `black.reformat_many` and its helpers
to `black.concurrency` which is now *only* imported if there's more
than one file to reformat. This way, running Black over a single file
is snappier
Here are the numbers before and after this patch running `python -m
black --version`:
- interpreted: 411 ms +- 9 ms -> 342 ms +- 7 ms: 1.20x faster
- compiled: 365 ms +- 15 ms -> 304 ms +- 7 ms: 1.20x faster
Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com>
Now PRs will run two diff-shades jobs, "preview-changes" which formats
all projects with preview=True, and "assert-no-changes" which formats
all projects with preview=False. The latter also fails if any changes
were made.
Pushes to main will only run "preview-changes"
Also the workflow_dispatch feature was dropped since it was
complicating everything for little gain.
I did this manually for the last few releases and I think it's going to be
helpful in the future too. Unfortunately this adds a little more work during
the release (sorry @cooperlees).
This change will also improve the merge conflict situation a bit, because
changes to different sections won't merge conflict.
For the last release, the sections were in a kind of random order. In the
template I put highlights and "Style" first because they're most important
to users, and alphabetized the rest.
At the moment, it's just a source of spurious CI failures and busywork
updating the configuration file.
Unlike diff-shades, it is run across many different platforms and
Python versions, but that doesn't seem essential. We already run unit
tests across platforms and versions.
I chose to leave the code around for now in case somebody is using it,
but CI will no longer run it.
Fixes#2506
``XDG_CACHE_HOME`` does not work on Windows. To allow for users to set a custom cache directory on all systems I added a new environment variable ``BLACK_CACHE_DIR`` to set the cache directory. The default remains the same so users will only notice a change if that environment variable is set.
The specific use case I have for this is I need to run black on in different processes at the same time. There is a race condition with the cache pickle file that made this rather difficult. A custom cache directory will remove the race condition.
I created ``get_cache_dir`` function in order to test the logic. This is only used to set the ``CACHE_DIR`` constant.
* Add stable tag process to release process documentation
- Add reasoning + step commands
* Bah - I ran the linter but forgot to commit
* Update docs/contributing/release_process.md
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
* Setup groundwork for release process docs
I'm using MyST for the index page since I like it more and it's easier
to work with.
* Fill in Release Process for black
* Apply suggestions from code review
Apply Jelle's grammar + typo fixes. I am a terrible only English speaker.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* Update release_process.md
Make lint happy via web UI.
* Move to contribution section and fix prettier
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
So these won't go out of date. This does mean the environment has be
setup a bit more carefully so the right version of the tool is used,
but thankfully the build environment is rebuilt on change on RTD anyway.
Also since the HTML docs are known to build fine, let's provide
downloadable HTMLzips of our docs.
This change needs RTD and GH to install Black with the [d] extra so
blackd's help can generated. While editing RTD's config file, let's
migrate the file to a non-deprecated filename.
Also I missed adding AUTHORS.md to the files key in the doc GHA config.
Commit history before merge:
* Replace references to master branch
* Update .flake8 to reference docs on RTD
We're moving away from GitHub as a documentation host to only RTD because
it's makes our lives easier creating good docs. I know this link is dead right now,
but it won't be once we release a new version with the documentation reorganization
changes (which should be soon!).
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
I know I know, this is the second reorganization of the docs. I'm not
saying the first one was bad or anything... but.. actually wait nah,
*it was bad*.
Anyway, welcome to probably my biggest commit. The main thing with this
reorganization was to introduce nesting to the documentation! Having
all of the docs be part of the main TOC was becoming too much. There
wasn't much room to expand either. Finally, the old setup required
a documentation generation step which was just annoying.
The goals of this reorganization was to:
1. Significantly restructure the docs to be discoverable and
understandable
2. Add room for further docs (like guides or contributing docs)
3. Get rid of the doc generation step (it was slow and frustrating)
4. Unblock other improvements and also just make contributing to the
docs easier
Another important change with this is that we are no longer using GitHub
as a documentation host. While GitHub does support Markdown based docs
actually pretty well, the lack of any features outside of GitHub Flavoured
Markdown is quite limiting. ReadTheDocs is just much better suited for
documentation. You can use reST, MyST, CommonMark, and all of their
great features like toctrees and admonitions.
Related to this change, we're adopting MyST as our flavour of Markdown.
MyST introduces neat syntax extensions to Markdown that pretty much
gives us the best of both worlds. The ease of use and simplicity of MD
and the flexibility and expressiveness of reST. Also recommonmark is
deprecated now. This switch was possible now we don't use GH as a docs
host. MyST docs have to be built to really be usable / pretty, so the MD
docs are going to look pretty bad on GH, but that's fine now!
Another thing that should be noted is that the README has been stripped
of most content since it was confusing. Users would read the README and
then think some feature or bug was fixed already and is available in a
release when in reality, they weren't. They were reading effectively
the latest docs without knowing.
See also: https://github.com/psf/black/issues/1759
FYI: CommonMark is a rationalized version of Markdown syntax
--
Commit history before merge:
* Switch to MyST-Parser + doc config cleanup
recommonmark is being deprecated in favour of MyST-Parser. This change
is welcomed, especially since MyST-Parser has syntax extensions for the
Commonmark standard. Effectively we get to use a language that's powerful
and expressive like ReST, but get the simplicity of Markdown.
The rest of this effort will be using some MyST features.
This reorganization efforts aims to remove as much duplication as possible.
The regeneration step once needed is gone, significantly simplifing our
Sphinx documentation configuration.
* Tell pipenv we replaced recommonmark for MyST-Parser
Also update `docs/requirements.txt`
* Delete all auto generated content
* Switch prettier for mdformat (plus a few plugins)
**FYI: THIS WAS EFFECTIVELY REVERTED, SEE THIRD TO LAST COMMIT**
prettier doesn't support MyST's syntax extensions which are going to be
used in this reorganization effort so we have to switch formatter.
Unfortanately mdformat's style is different from prettier's so time to
reformat the whole repo too.
We're excluding .github/ISSUE_TEMPLATE because I have no idea whether
its changes are safe, so let's play it safe.
* Fix the heading levels in CHANGES.md + a link
MyST-Parser / sphinx's linkcheck complains otherwise.
* Move reference docs into a docs/contributing dir
They're for contributors of Black anyway. Also added a note in the
summary document warning about the lack of attention the reference has
been dealing with.
* Rewrite and setup the new landing page + main TOC
- add some more detail about Black's beta status
- add licensing info
- add external links in the main TOC for GitHub, PyPI, and IRC
- prepare main TOC for new structure
* Break out AUTHORS into its own file
Not only was the AUTHORS list quite long, this makes it easy to include
it in the Sphinx docs with just a simple symlink.
* Add license to docs via a simple include
Yes the document is orphaned but it is linked to in the landing page
(docs/index.rst).
* Add "The Black Code Style" section
This mostly was a restructuring commit, there has been a few updates but
not many. The main goal was to split "current style" and "planned
changes to the style that haven't happened yet" to avoid confusion.
* Add "Getting Started" page
This is basically a quick start + even more. This commit is certainly
one of most creatively involved in this effort.
* Add "Usage and Configuration" section
This commit was as much restructuring as new content. Instead of being
in one giant file, usage and configuration documentation can expand
without bloating a single file.
* Add "Integrations" section
Just a restructuring commit ...
* Add "Guides" section
This is a promising area of documentation that could easily grow in the
future, let's prepare for that!
* Add "Contributing" section
This is also another area that I expect to see significant growth in.
Contributors to Black could definitely do with some more specific docs
that clears up certain parts of our slightly confusing project (it's
only confusing because we're getting big and old!).
* Rewrite CONTRIBUTING.md to just point to RTD
* Rewrite README.md to delegate most info to RTD
* Address feedback + a lot of corrections and edits
I know I said I wanted to do these after landing this but given there's
going to be no time between this being merged and a release getting
pushed, I want these changes to make it in.
- drop the number flag for mdformat - to reduce diffs, see also:
https://mdformat.readthedocs.io/en/stable/users/style.html#ordered-lists
- the GH issue templates should be safe by mdformat, so get rid of the
exclude
- clarify our configuration position - i.e. stop claiming we don't have
many options, instead say we want as little formatting knobs as
possible
- lots and lots of punctuation, spelling, and grammar corrections (thanks
Jelle!)
- use RTD as the source for the CHANGELOG too
- visual style cleanups
- add docs about our .gitignore behaviour
- expand GHA Action docs
- clarify we want the PR number in the CHANGELOG entry
- claify Black's behaviour for with statements post Python 3.9
- italicize a bunch of "Black"s
Thank you goes to Jelle, Taneli (hukkinj1 on GH), Felix
(felix-hilden on GH), and Wouter (wbolster on GH) for the feedback!
* Merge remote-tracking branch 'upstream/master' into reorganize-docs-v2
merge conflicts suck, although these ones weren't too bad.
* Add changelog entry + fix merge conflict resolution error
I consider this important enough to be worthy of a changelog entry :)
* Merge branch 'master' into reorganize-docs-v2
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* Actually let's continue using prettier
Prettier works fine for all of the default MyST syntax so let's not
rock the boat as much. Dropping the mdformat commit was merge-conflict
filled so here's additional commit instead.
* Address Cooper's, Taneli's, and Jelle's feedback
Lots of wording improvements by Cooper. Taneli suggested to disable the
enabled by default MyST syntax not supported by Prettier and I agreed.
And Jelle found one more spelling error!
* More minor fixes
