Remove Python 2 support (#2740)

*blib2to3's support was left untouched because: 1) I don't want to touch
parsing machinery, and 2) it'll allow us to provide a more useful error
message if someone does try to format Python 2 code.
This commit is contained in:
Richard Si 2022-01-10 07:16:30 -05:00 committed by GitHub
parent e64949ee69
commit e401b6bb1e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
27 changed files with 73 additions and 391 deletions

View File

@ -16,7 +16,7 @@ current development version. To confirm this, you have three options:
3. Or run _Black_ on your machine: 3. Or run _Black_ on your machine:
- create a new virtualenv (make sure it's the same Python version); - create a new virtualenv (make sure it's the same Python version);
- clone this repository; - clone this repository;
- run `pip install -e .[d,python2]`; - run `pip install -e .[d]`;
- run `pip install -r test_requirements.txt` - run `pip install -r test_requirements.txt`
- make sure it's sane by running `python -m pytest`; and - make sure it's sane by running `python -m pytest`; and
- run `black` like you did last time. - run `black` like you did last time.

View File

@ -4,6 +4,7 @@
### _Black_ ### _Black_
- **Remove Python 2 support** (#2740)
- Do not accept bare carriage return line endings in pyproject.toml (#2408) - Do not accept bare carriage return line endings in pyproject.toml (#2408)
- Improve error message for invalid regular expression (#2678) - Improve error message for invalid regular expression (#2678)
- Improve error message when parsing fails during AST safety check by embedding the - Improve error message when parsing fails during AST safety check by embedding the

View File

@ -40,9 +40,7 @@ Try it out now using the [Black Playground](https://black.vercel.app). Watch the
### Installation ### Installation
_Black_ can be installed by running `pip install black`. It requires Python 3.6.2+ to _Black_ can be installed by running `pip install black`. It requires Python 3.6.2+ to
run. If you want to format Python 2 code as well, install with run. If you want to format Jupyter Notebooks, install with `pip install black[jupyter]`.
`pip install black[python2]`. If you want to format Jupyter Notebooks, install with
`pip install black[jupyter]`.
If you can't wait for the latest _hotness_ and want to install from GitHub, use: If you can't wait for the latest _hotness_ and want to install from GitHub, use:

View File

@ -14,7 +14,7 @@
run([sys.executable, "-m", "venv", str(ENV_PATH)], check=True) run([sys.executable, "-m", "venv", str(ENV_PATH)], check=True)
req = "black[colorama,python2]" req = "black[colorama]"
if VERSION: if VERSION:
req += f"=={VERSION}" req += f"=={VERSION}"
pip_proc = run( pip_proc = run(

View File

@ -75,16 +75,7 @@ disabled-by-default counterpart W504. E203 should be disabled while changes are
## Does Black support Python 2? ## Does Black support Python 2?
```{warning} Support for formatting Python 2 code was removed in version 22.0.
Python 2 support has been deprecated since 21.10b0.
This support will be dropped in the first stable release, expected for January 2022.
See [The Black Code Style](the_black_code_style/index.rst) for details.
```
For formatting, yes! [Install](getting_started.md#installation) with the `python2` extra
to format Python 2 files too! In terms of running _Black_ though, Python 3.6 or newer is
required.
## Why does my linter or typechecker complain after I format my code? ## Why does my linter or typechecker complain after I format my code?
@ -96,8 +87,7 @@ codebase with _Black_.
## Can I run Black with PyPy? ## Can I run Black with PyPy?
Yes, there is support for PyPy 3.7 and higher. You cannot format Python 2 files under Yes, there is support for PyPy 3.7 and higher.
PyPy, because PyPy's inbuilt ast module does not support this.
## Why does Black not detect syntax errors in my code? ## Why does Black not detect syntax errors in my code?

View File

@ -17,9 +17,7 @@ Also, you can try out _Black_ online for minimal fuss on the
## Installation ## Installation
_Black_ can be installed by running `pip install black`. It requires Python 3.6.2+ to _Black_ can be installed by running `pip install black`. It requires Python 3.6.2+ to
run, but can format Python 2 code too. Python 2 support needs the `typed_ast` run. If you want to format Jupyter Notebooks, install with `pip install black[jupyter]`.
dependency, which be installed with `pip install black[python2]`. If you want to format
Jupyter Notebooks, install with `pip install black[jupyter]`.
If you can't wait for the latest _hotness_ and want to install from GitHub, use: If you can't wait for the latest _hotness_ and want to install from GitHub, use:

View File

@ -8,8 +8,8 @@ environment. Great for enforcing that your code matches the _Black_ code style.
This action is known to support all GitHub-hosted runner OSes. In addition, only This action is known to support all GitHub-hosted runner OSes. In addition, only
published versions of _Black_ are supported (i.e. whatever is available on PyPI). published versions of _Black_ are supported (i.e. whatever is available on PyPI).
Finally, this action installs _Black_ with both the `colorama` and `python2` extras so Finally, this action installs _Black_ with the `colorama` extra so the `--color` flag
the `--color` flag and formatting Python 2 code are supported. should work fine.
## Usage ## Usage

View File

@ -281,8 +281,7 @@ removed.
_Black_ standardizes most numeric literals to use lowercase letters for the syntactic _Black_ standardizes most numeric literals to use lowercase letters for the syntactic
parts and uppercase letters for the digits themselves: `0xAB` instead of `0XAB` and parts and uppercase letters for the digits themselves: `0xAB` instead of `0XAB` and
`1e10` instead of `1E10`. Python 2 long literals are styled as `2L` instead of `2l` to `1e10` instead of `1E10`.
avoid confusion between `l` and `1`.
### Line breaks & binary operators ### Line breaks & binary operators

View File

@ -29,7 +29,6 @@ build-backend = "setuptools.build_meta"
[tool.pytest.ini_options] [tool.pytest.ini_options]
# Option below requires `tests/optional.py` # Option below requires `tests/optional.py`
optional-tests = [ optional-tests = [
"no_python2: run when `python2` extra NOT installed",
"no_blackd: run when `d` extra NOT installed", "no_blackd: run when `d` extra NOT installed",
"no_jupyter: run when `jupyter` extra NOT installed", "no_jupyter: run when `jupyter` extra NOT installed",
] ]

View File

@ -112,7 +112,6 @@ def find_python_files(base: Path) -> List[Path]:
extras_require={ extras_require={
"d": ["aiohttp>=3.7.4"], "d": ["aiohttp>=3.7.4"],
"colorama": ["colorama>=0.4.3"], "colorama": ["colorama>=0.4.3"],
"python2": ["typed-ast>=1.4.3"],
"uvloop": ["uvloop>=0.15.2"], "uvloop": ["uvloop>=0.15.2"],
"jupyter": ["ipython>=7.8.0", "tokenize-rt>=3.2.0"], "jupyter": ["ipython>=7.8.0", "tokenize-rt>=3.2.0"],
}, },

View File

@ -1083,20 +1083,8 @@ def f(
else: else:
versions = detect_target_versions(src_node, future_imports=future_imports) versions = detect_target_versions(src_node, future_imports=future_imports)
# TODO: fully drop support and this code hopefully in January 2022 :D
if TargetVersion.PY27 in mode.target_versions or versions == {TargetVersion.PY27}:
msg = (
"DEPRECATION: Python 2 support will be removed in the first stable release "
"expected in January 2022."
)
err(msg, fg="yellow", bold=True)
normalize_fmt_off(src_node) normalize_fmt_off(src_node)
lines = LineGenerator( lines = LineGenerator(mode=mode)
mode=mode,
remove_u_prefix="unicode_literals" in future_imports
or supports_feature(versions, Feature.UNICODE_LITERALS),
)
elt = EmptyLineTracker(is_pyi=mode.is_pyi) elt = EmptyLineTracker(is_pyi=mode.is_pyi)
empty_line = Line(mode=mode) empty_line = Line(mode=mode)
after = 0 after = 0
@ -1166,14 +1154,6 @@ def get_features_used( # noqa: C901
assert isinstance(n, Leaf) assert isinstance(n, Leaf)
if "_" in n.value: if "_" in n.value:
features.add(Feature.NUMERIC_UNDERSCORES) features.add(Feature.NUMERIC_UNDERSCORES)
elif n.value.endswith(("L", "l")):
# Python 2: 10L
features.add(Feature.LONG_INT_LITERAL)
elif len(n.value) >= 2 and n.value[0] == "0" and n.value[1].isdigit():
# Python 2: 0123; 00123; ...
if not all(char == "0" for char in n.value):
# although we don't want to match 0000 or similar
features.add(Feature.OCTAL_INT_LITERAL)
elif n.type == token.SLASH: elif n.type == token.SLASH:
if n.parent and n.parent.type in { if n.parent and n.parent.type in {
@ -1226,32 +1206,6 @@ def get_features_used( # noqa: C901
): ):
features.add(Feature.ANN_ASSIGN_EXTENDED_RHS) features.add(Feature.ANN_ASSIGN_EXTENDED_RHS)
# Python 2 only features (for its deprecation) except for integers, see above
elif n.type == syms.print_stmt:
features.add(Feature.PRINT_STMT)
elif n.type == syms.exec_stmt:
features.add(Feature.EXEC_STMT)
elif n.type == syms.tfpdef:
# def set_position((x, y), value):
# ...
features.add(Feature.AUTOMATIC_PARAMETER_UNPACKING)
elif n.type == syms.except_clause:
# try:
# ...
# except Exception, err:
# ...
if len(n.children) >= 4:
if n.children[-2].type == token.COMMA:
features.add(Feature.COMMA_STYLE_EXCEPT)
elif n.type == syms.raise_stmt:
# raise Exception, "msg"
if len(n.children) >= 4:
if n.children[-2].type == token.COMMA:
features.add(Feature.COMMA_STYLE_RAISE)
elif n.type == token.BACKQUOTE:
# `i'm surprised this ever existed`
features.add(Feature.BACKQUOTE_REPR)
return features return features

View File

@ -48,9 +48,8 @@ class LineGenerator(Visitor[Line]):
in ways that will no longer stringify to valid Python code on the tree. in ways that will no longer stringify to valid Python code on the tree.
""" """
def __init__(self, mode: Mode, remove_u_prefix: bool = False) -> None: def __init__(self, mode: Mode) -> None:
self.mode = mode self.mode = mode
self.remove_u_prefix = remove_u_prefix
self.current_line: Line self.current_line: Line
self.__post_init__() self.__post_init__()
@ -92,9 +91,7 @@ def visit_default(self, node: LN) -> Iterator[Line]:
normalize_prefix(node, inside_brackets=any_open_brackets) normalize_prefix(node, inside_brackets=any_open_brackets)
if self.mode.string_normalization and node.type == token.STRING: if self.mode.string_normalization and node.type == token.STRING:
node.value = normalize_string_prefix( node.value = normalize_string_prefix(node.value)
node.value, remove_u_prefix=self.remove_u_prefix
)
node.value = normalize_string_quotes(node.value) node.value = normalize_string_quotes(node.value)
if node.type == token.NUMBER: if node.type == token.NUMBER:
normalize_numeric_literal(node) normalize_numeric_literal(node)
@ -236,7 +233,7 @@ def visit_STRING(self, leaf: Leaf) -> Iterator[Line]:
if is_docstring(leaf) and "\\\n" not in leaf.value: if is_docstring(leaf) and "\\\n" not in leaf.value:
# We're ignoring docstrings with backslash newline escapes because changing # We're ignoring docstrings with backslash newline escapes because changing
# indentation of those changes the AST representation of the code. # indentation of those changes the AST representation of the code.
docstring = normalize_string_prefix(leaf.value, self.remove_u_prefix) docstring = normalize_string_prefix(leaf.value)
prefix = get_string_prefix(docstring) prefix = get_string_prefix(docstring)
docstring = docstring[len(prefix) :] # Remove the prefix docstring = docstring[len(prefix) :] # Remove the prefix
quote_char = docstring[0] quote_char = docstring[0]

View File

@ -20,7 +20,6 @@
class TargetVersion(Enum): class TargetVersion(Enum):
PY27 = 2
PY33 = 3 PY33 = 3
PY34 = 4 PY34 = 4
PY35 = 5 PY35 = 5
@ -30,13 +29,8 @@ class TargetVersion(Enum):
PY39 = 9 PY39 = 9
PY310 = 10 PY310 = 10
def is_python2(self) -> bool:
return self is TargetVersion.PY27
class Feature(Enum): class Feature(Enum):
# All string literals are unicode
UNICODE_LITERALS = 1
F_STRINGS = 2 F_STRINGS = 2
NUMERIC_UNDERSCORES = 3 NUMERIC_UNDERSCORES = 3
TRAILING_COMMA_IN_CALL = 4 TRAILING_COMMA_IN_CALL = 4
@ -56,16 +50,6 @@ class Feature(Enum):
# __future__ flags # __future__ flags
FUTURE_ANNOTATIONS = 51 FUTURE_ANNOTATIONS = 51
# temporary for Python 2 deprecation
PRINT_STMT = 200
EXEC_STMT = 201
AUTOMATIC_PARAMETER_UNPACKING = 202
COMMA_STYLE_EXCEPT = 203
COMMA_STYLE_RAISE = 204
LONG_INT_LITERAL = 205
OCTAL_INT_LITERAL = 206
BACKQUOTE_REPR = 207
FUTURE_FLAG_TO_FEATURE: Final = { FUTURE_FLAG_TO_FEATURE: Final = {
"annotations": Feature.FUTURE_ANNOTATIONS, "annotations": Feature.FUTURE_ANNOTATIONS,
@ -73,26 +57,10 @@ class Feature(Enum):
VERSION_TO_FEATURES: Dict[TargetVersion, Set[Feature]] = { VERSION_TO_FEATURES: Dict[TargetVersion, Set[Feature]] = {
TargetVersion.PY27: { TargetVersion.PY33: {Feature.ASYNC_IDENTIFIERS},
Feature.ASYNC_IDENTIFIERS, TargetVersion.PY34: {Feature.ASYNC_IDENTIFIERS},
Feature.PRINT_STMT, TargetVersion.PY35: {Feature.TRAILING_COMMA_IN_CALL, Feature.ASYNC_IDENTIFIERS},
Feature.EXEC_STMT,
Feature.AUTOMATIC_PARAMETER_UNPACKING,
Feature.COMMA_STYLE_EXCEPT,
Feature.COMMA_STYLE_RAISE,
Feature.LONG_INT_LITERAL,
Feature.OCTAL_INT_LITERAL,
Feature.BACKQUOTE_REPR,
},
TargetVersion.PY33: {Feature.UNICODE_LITERALS, Feature.ASYNC_IDENTIFIERS},
TargetVersion.PY34: {Feature.UNICODE_LITERALS, Feature.ASYNC_IDENTIFIERS},
TargetVersion.PY35: {
Feature.UNICODE_LITERALS,
Feature.TRAILING_COMMA_IN_CALL,
Feature.ASYNC_IDENTIFIERS,
},
TargetVersion.PY36: { TargetVersion.PY36: {
Feature.UNICODE_LITERALS,
Feature.F_STRINGS, Feature.F_STRINGS,
Feature.NUMERIC_UNDERSCORES, Feature.NUMERIC_UNDERSCORES,
Feature.TRAILING_COMMA_IN_CALL, Feature.TRAILING_COMMA_IN_CALL,
@ -100,7 +68,6 @@ class Feature(Enum):
Feature.ASYNC_IDENTIFIERS, Feature.ASYNC_IDENTIFIERS,
}, },
TargetVersion.PY37: { TargetVersion.PY37: {
Feature.UNICODE_LITERALS,
Feature.F_STRINGS, Feature.F_STRINGS,
Feature.NUMERIC_UNDERSCORES, Feature.NUMERIC_UNDERSCORES,
Feature.TRAILING_COMMA_IN_CALL, Feature.TRAILING_COMMA_IN_CALL,
@ -109,7 +76,6 @@ class Feature(Enum):
Feature.FUTURE_ANNOTATIONS, Feature.FUTURE_ANNOTATIONS,
}, },
TargetVersion.PY38: { TargetVersion.PY38: {
Feature.UNICODE_LITERALS,
Feature.F_STRINGS, Feature.F_STRINGS,
Feature.NUMERIC_UNDERSCORES, Feature.NUMERIC_UNDERSCORES,
Feature.TRAILING_COMMA_IN_CALL, Feature.TRAILING_COMMA_IN_CALL,
@ -122,7 +88,6 @@ class Feature(Enum):
Feature.ANN_ASSIGN_EXTENDED_RHS, Feature.ANN_ASSIGN_EXTENDED_RHS,
}, },
TargetVersion.PY39: { TargetVersion.PY39: {
Feature.UNICODE_LITERALS,
Feature.F_STRINGS, Feature.F_STRINGS,
Feature.NUMERIC_UNDERSCORES, Feature.NUMERIC_UNDERSCORES,
Feature.TRAILING_COMMA_IN_CALL, Feature.TRAILING_COMMA_IN_CALL,
@ -136,7 +101,6 @@ class Feature(Enum):
Feature.ANN_ASSIGN_EXTENDED_RHS, Feature.ANN_ASSIGN_EXTENDED_RHS,
}, },
TargetVersion.PY310: { TargetVersion.PY310: {
Feature.UNICODE_LITERALS,
Feature.F_STRINGS, Feature.F_STRINGS,
Feature.NUMERIC_UNDERSCORES, Feature.NUMERIC_UNDERSCORES,
Feature.TRAILING_COMMA_IN_CALL, Feature.TRAILING_COMMA_IN_CALL,

View File

@ -259,16 +259,6 @@ def whitespace(leaf: Leaf, *, complex_subscript: bool) -> str: # noqa: C901
): ):
return NO return NO
elif (
prevp.type == token.RIGHTSHIFT
and prevp.parent
and prevp.parent.type == syms.shift_expr
and prevp.prev_sibling
and is_name_token(prevp.prev_sibling)
and prevp.prev_sibling.value == "print"
):
# Python 2 print chevron
return NO
elif prevp.type == token.AT and p.parent and p.parent.type == syms.decorator: elif prevp.type == token.AT and p.parent and p.parent.type == syms.decorator:
# no space in decorators # no space in decorators
return NO return NO

View File

@ -25,13 +25,10 @@ def format_scientific_notation(text: str) -> str:
return f"{before}e{sign}{after}" return f"{before}e{sign}{after}"
def format_long_or_complex_number(text: str) -> str: def format_complex_number(text: str) -> str:
"""Formats a long or complex string like `10L` or `10j`""" """Formats a complex string like `10j`"""
number = text[:-1] number = text[:-1]
suffix = text[-1] suffix = text[-1]
# Capitalize in "2L" because "l" looks too similar to "1".
if suffix == "l":
suffix = "L"
return f"{format_float_or_int_string(number)}{suffix}" return f"{format_float_or_int_string(number)}{suffix}"
@ -47,9 +44,7 @@ def format_float_or_int_string(text: str) -> str:
def normalize_numeric_literal(leaf: Leaf) -> None: def normalize_numeric_literal(leaf: Leaf) -> None:
"""Normalizes numeric (float, int, and complex) literals. """Normalizes numeric (float, int, and complex) literals.
All letters used in the representation are normalized to lowercase (except All letters used in the representation are normalized to lowercase."""
in Python 2 long literals).
"""
text = leaf.value.lower() text = leaf.value.lower()
if text.startswith(("0o", "0b")): if text.startswith(("0o", "0b")):
# Leave octal and binary literals alone. # Leave octal and binary literals alone.
@ -58,8 +53,8 @@ def normalize_numeric_literal(leaf: Leaf) -> None:
text = format_hex(text) text = format_hex(text)
elif "e" in text: elif "e" in text:
text = format_scientific_notation(text) text = format_scientific_notation(text)
elif text.endswith(("j", "l")): elif text.endswith("j"):
text = format_long_or_complex_number(text) text = format_complex_number(text)
else: else:
text = format_float_or_int_string(text) text = format_float_or_int_string(text)
leaf.value = text leaf.value = text

View File

@ -4,7 +4,7 @@
import ast import ast
import platform import platform
import sys import sys
from typing import Any, AnyStr, Iterable, Iterator, List, Set, Tuple, Type, Union from typing import Any, Iterable, Iterator, List, Set, Tuple, Type, Union
if sys.version_info < (3, 8): if sys.version_info < (3, 8):
from typing_extensions import Final from typing_extensions import Final
@ -23,12 +23,11 @@
from black.nodes import syms from black.nodes import syms
ast3: Any ast3: Any
ast27: Any
_IS_PYPY = platform.python_implementation() == "PyPy" _IS_PYPY = platform.python_implementation() == "PyPy"
try: try:
from typed_ast import ast3, ast27 from typed_ast import ast3
except ImportError: except ImportError:
# Either our python version is too low, or we're on pypy # Either our python version is too low, or we're on pypy
if sys.version_info < (3, 7) or (sys.version_info < (3, 8) and not _IS_PYPY): if sys.version_info < (3, 7) or (sys.version_info < (3, 8) and not _IS_PYPY):
@ -40,12 +39,11 @@
) )
sys.exit(1) sys.exit(1)
else: else:
ast3 = ast27 = ast ast3 = ast
PY310_HINT: Final[ PY310_HINT: Final = "Consider using --target-version py310 to parse Python 3.10 code."
str PY2_HINT: Final = "Python 2 support was removed in version 22.0."
] = "Consider using --target-version py310 to parse Python 3.10 code."
class InvalidInput(ValueError): class InvalidInput(ValueError):
@ -60,22 +58,8 @@ def get_grammars(target_versions: Set[TargetVersion]) -> List[Grammar]:
pygram.python_grammar_no_print_statement_no_exec_statement_async_keywords, pygram.python_grammar_no_print_statement_no_exec_statement_async_keywords,
# Python 3.0-3.6 # Python 3.0-3.6
pygram.python_grammar_no_print_statement_no_exec_statement, pygram.python_grammar_no_print_statement_no_exec_statement,
# Python 2.7 with future print_function import
pygram.python_grammar_no_print_statement,
# Python 2.7
pygram.python_grammar,
] ]
if all(version.is_python2() for version in target_versions):
# Python 2-only code, so try Python 2 grammars.
return [
# Python 2.7 with future print_function import
pygram.python_grammar_no_print_statement,
# Python 2.7
pygram.python_grammar,
]
# Python 3-compatible code, so only try Python 3 grammar.
grammars = [] grammars = []
if supports_feature(target_versions, Feature.PATTERN_MATCHING): if supports_feature(target_versions, Feature.PATTERN_MATCHING):
# Python 3.10+ # Python 3.10+
@ -129,6 +113,14 @@ def lib2to3_parse(src_txt: str, target_versions: Iterable[TargetVersion] = ()) -
original_msg = exc.args[0] original_msg = exc.args[0]
msg = f"{original_msg}\n{PY310_HINT}" msg = f"{original_msg}\n{PY310_HINT}"
raise InvalidInput(msg) from None raise InvalidInput(msg) from None
if matches_grammar(src_txt, pygram.python_grammar) or matches_grammar(
src_txt, pygram.python_grammar_no_print_statement
):
original_msg = exc.args[0]
msg = f"{original_msg}\n{PY2_HINT}"
raise InvalidInput(msg) from None
raise exc from None raise exc from None
if isinstance(result, Leaf): if isinstance(result, Leaf):
@ -154,7 +146,7 @@ def lib2to3_unparse(node: Node) -> str:
def parse_single_version( def parse_single_version(
src: str, version: Tuple[int, int] src: str, version: Tuple[int, int]
) -> Union[ast.AST, ast3.AST, ast27.AST]: ) -> Union[ast.AST, ast3.AST]:
filename = "<unknown>" filename = "<unknown>"
# typed_ast is needed because of feature version limitations in the builtin ast # typed_ast is needed because of feature version limitations in the builtin ast
if sys.version_info >= (3, 8) and version >= (3,): if sys.version_info >= (3, 8) and version >= (3,):
@ -164,18 +156,13 @@ def parse_single_version(
return ast3.parse(src, filename) return ast3.parse(src, filename)
else: else:
return ast3.parse(src, filename, feature_version=version[1]) return ast3.parse(src, filename, feature_version=version[1])
elif version == (2, 7):
return ast27.parse(src)
raise AssertionError("INTERNAL ERROR: Tried parsing unsupported Python version!") raise AssertionError("INTERNAL ERROR: Tried parsing unsupported Python version!")
def parse_ast(src: str) -> Union[ast.AST, ast3.AST, ast27.AST]: def parse_ast(src: str) -> Union[ast.AST, ast3.AST]:
# TODO: support Python 4+ ;) # TODO: support Python 4+ ;)
versions = [(3, minor) for minor in range(3, sys.version_info[1] + 1)] versions = [(3, minor) for minor in range(3, sys.version_info[1] + 1)]
if ast27.__name__ != "ast":
versions.append((2, 7))
first_error = "" first_error = ""
for version in sorted(versions, reverse=True): for version in sorted(versions, reverse=True):
try: try:
@ -188,22 +175,19 @@ def parse_ast(src: str) -> Union[ast.AST, ast3.AST, ast27.AST]:
ast3_AST: Final[Type[ast3.AST]] = ast3.AST ast3_AST: Final[Type[ast3.AST]] = ast3.AST
ast27_AST: Final[Type[ast27.AST]] = ast27.AST
def _normalize(lineend: AnyStr, value: AnyStr) -> AnyStr: def _normalize(lineend: str, value: str) -> str:
# To normalize, we strip any leading and trailing space from # To normalize, we strip any leading and trailing space from
# each line... # each line...
stripped: List[AnyStr] = [i.strip() for i in value.splitlines()] stripped: List[str] = [i.strip() for i in value.splitlines()]
normalized = lineend.join(stripped) normalized = lineend.join(stripped)
# ...and remove any blank lines at the beginning and end of # ...and remove any blank lines at the beginning and end of
# the whole string # the whole string
return normalized.strip() return normalized.strip()
def stringify_ast( def stringify_ast(node: Union[ast.AST, ast3.AST], depth: int = 0) -> Iterator[str]:
node: Union[ast.AST, ast3.AST, ast27.AST], depth: int = 0
) -> Iterator[str]:
"""Simple visitor generating strings to compare ASTs by content.""" """Simple visitor generating strings to compare ASTs by content."""
node = fixup_ast_constants(node) node = fixup_ast_constants(node)
@ -215,7 +199,7 @@ def stringify_ast(
# TypeIgnore will not be present using pypy < 3.8, so need for this # TypeIgnore will not be present using pypy < 3.8, so need for this
if not (_IS_PYPY and sys.version_info < (3, 8)): if not (_IS_PYPY and sys.version_info < (3, 8)):
# TypeIgnore has only one field 'lineno' which breaks this comparison # TypeIgnore has only one field 'lineno' which breaks this comparison
type_ignore_classes = (ast3.TypeIgnore, ast27.TypeIgnore) type_ignore_classes = (ast3.TypeIgnore,)
if sys.version_info >= (3, 8): if sys.version_info >= (3, 8):
type_ignore_classes += (ast.TypeIgnore,) type_ignore_classes += (ast.TypeIgnore,)
if isinstance(node, type_ignore_classes): if isinstance(node, type_ignore_classes):
@ -234,40 +218,34 @@ def stringify_ast(
# parentheses and they change the AST. # parentheses and they change the AST.
if ( if (
field == "targets" field == "targets"
and isinstance(node, (ast.Delete, ast3.Delete, ast27.Delete)) and isinstance(node, (ast.Delete, ast3.Delete))
and isinstance(item, (ast.Tuple, ast3.Tuple, ast27.Tuple)) and isinstance(item, (ast.Tuple, ast3.Tuple))
): ):
for item in item.elts: for item in item.elts:
yield from stringify_ast(item, depth + 2) yield from stringify_ast(item, depth + 2)
elif isinstance(item, (ast.AST, ast3.AST, ast27.AST)): elif isinstance(item, (ast.AST, ast3.AST)):
yield from stringify_ast(item, depth + 2) yield from stringify_ast(item, depth + 2)
# Note that we are referencing the typed-ast ASTs via global variables and not # Note that we are referencing the typed-ast ASTs via global variables and not
# direct module attribute accesses because that breaks mypyc. It's probably # direct module attribute accesses because that breaks mypyc. It's probably
# something to do with the ast3 / ast27 variables being marked as Any leading # something to do with the ast3 variables being marked as Any leading
# mypy to think this branch is always taken, leaving the rest of the code # mypy to think this branch is always taken, leaving the rest of the code
# unanalyzed. Tighting up the types for the typed-ast AST types avoids the # unanalyzed. Tighting up the types for the typed-ast AST types avoids the
# mypyc crash. # mypyc crash.
elif isinstance(value, (ast.AST, ast3_AST, ast27_AST)): elif isinstance(value, (ast.AST, ast3_AST)):
yield from stringify_ast(value, depth + 2) yield from stringify_ast(value, depth + 2)
else: else:
# Constant strings may be indented across newlines, if they are # Constant strings may be indented across newlines, if they are
# docstrings; fold spaces after newlines when comparing. Similarly, # docstrings; fold spaces after newlines when comparing. Similarly,
# trailing and leading space may be removed. # trailing and leading space may be removed.
# Note that when formatting Python 2 code, at least with Windows
# line-endings, docstrings can end up here as bytes instead of
# str so make sure that we handle both cases.
if ( if (
isinstance(node, ast.Constant) isinstance(node, ast.Constant)
and field == "value" and field == "value"
and isinstance(value, (str, bytes)) and isinstance(value, str)
): ):
if isinstance(value, str): normalized = _normalize("\n", value)
normalized: Union[str, bytes] = _normalize("\n", value)
else:
normalized = _normalize(b"\n", value)
else: else:
normalized = value normalized = value
yield f"{' ' * (depth+2)}{normalized!r}, # {value.__class__.__name__}" yield f"{' ' * (depth+2)}{normalized!r}, # {value.__class__.__name__}"
@ -275,14 +253,12 @@ def stringify_ast(
yield f"{' ' * depth}) # /{node.__class__.__name__}" yield f"{' ' * depth}) # /{node.__class__.__name__}"
def fixup_ast_constants( def fixup_ast_constants(node: Union[ast.AST, ast3.AST]) -> Union[ast.AST, ast3.AST]:
node: Union[ast.AST, ast3.AST, ast27.AST]
) -> Union[ast.AST, ast3.AST, ast27.AST]:
"""Map ast nodes deprecated in 3.8 to Constant.""" """Map ast nodes deprecated in 3.8 to Constant."""
if isinstance(node, (ast.Str, ast3.Str, ast27.Str, ast.Bytes, ast3.Bytes)): if isinstance(node, (ast.Str, ast3.Str, ast.Bytes, ast3.Bytes)):
return ast.Constant(value=node.s) return ast.Constant(value=node.s)
if isinstance(node, (ast.Num, ast3.Num, ast27.Num)): if isinstance(node, (ast.Num, ast3.Num)):
return ast.Constant(value=node.n) return ast.Constant(value=node.n)
if isinstance(node, (ast.NameConstant, ast3.NameConstant)): if isinstance(node, (ast.NameConstant, ast3.NameConstant)):

View File

@ -138,17 +138,17 @@ def assert_is_leaf_string(string: str) -> None:
), f"{set(string[:quote_idx])} is NOT a subset of {set(STRING_PREFIX_CHARS)}." ), f"{set(string[:quote_idx])} is NOT a subset of {set(STRING_PREFIX_CHARS)}."
def normalize_string_prefix(s: str, remove_u_prefix: bool = False) -> str: def normalize_string_prefix(s: str) -> str:
"""Make all string prefixes lowercase. """Make all string prefixes lowercase."""
If remove_u_prefix is given, also removes any u prefix from the string.
"""
match = STRING_PREFIX_RE.match(s) match = STRING_PREFIX_RE.match(s)
assert match is not None, f"failed to match string {s!r}" assert match is not None, f"failed to match string {s!r}"
orig_prefix = match.group(1) orig_prefix = match.group(1)
new_prefix = orig_prefix.replace("F", "f").replace("B", "b").replace("U", "u") new_prefix = (
if remove_u_prefix: orig_prefix.replace("F", "f")
new_prefix = new_prefix.replace("u", "") .replace("B", "b")
.replace("U", "")
.replace("u", "")
)
return f"{new_prefix}{match.group(2)}" return f"{new_prefix}{match.group(2)}"

View File

@ -149,17 +149,6 @@
"long_checkout": false, "long_checkout": false,
"py_versions": ["all"] "py_versions": ["all"]
}, },
"sqlalchemy": {
"cli_arguments": [
"--experimental-string-processing",
"--extend-exclude",
"/test/orm/test_relationship_criteria.py"
],
"expect_formatting_changes": true,
"git_clone_url": "https://github.com/sqlalchemy/sqlalchemy.git",
"long_checkout": false,
"py_versions": ["all"]
},
"tox": { "tox": {
"cli_arguments": ["--experimental-string-processing"], "cli_arguments": ["--experimental-string-processing"],
"expect_formatting_changes": true, "expect_formatting_changes": true,

View File

@ -174,10 +174,8 @@ def parse_python_variant_header(value: str) -> Tuple[bool, Set[black.TargetVersi
raise InvalidVariantHeader("major version must be 2 or 3") raise InvalidVariantHeader("major version must be 2 or 3")
if len(rest) > 0: if len(rest) > 0:
minor = int(rest[0]) minor = int(rest[0])
if major == 2 and minor != 7: if major == 2:
raise InvalidVariantHeader( raise InvalidVariantHeader("Python 2 is not supported")
"minor version must be 7 for Python 2"
)
else: else:
# Default to lowest supported minor version. # Default to lowest supported minor version.
minor = 7 if major == 2 else 3 minor = 7 if major == 2 else 3

View File

@ -1,16 +0,0 @@
#!/usr/bin/env python2.7
x = 123456789L
x = 123456789l
x = 123456789
x = 0xb1acc
# output
#!/usr/bin/env python2.7
x = 123456789L
x = 123456789L
x = 123456789
x = 0xB1ACC

View File

@ -1,33 +0,0 @@
#!/usr/bin/env python2
import sys
print >> sys.stderr , "Warning:" ,
print >> sys.stderr , "this is a blast from the past."
print >> sys.stderr , "Look, a repr:", `sys`
def function((_globals, _locals)):
exec ur"print 'hi from exec!'" in _globals, _locals
function((globals(), locals()))
# output
#!/usr/bin/env python2
import sys
print >>sys.stderr, "Warning:",
print >>sys.stderr, "this is a blast from the past."
print >>sys.stderr, "Look, a repr:", ` sys `
def function((_globals, _locals)):
exec ur"print 'hi from exec!'" in _globals, _locals
function((globals(), locals()))

View File

@ -1,16 +0,0 @@
#!/usr/bin/env python2
from __future__ import print_function
print('hello')
print(u'hello')
print(a, file=sys.stderr)
# output
#!/usr/bin/env python2
from __future__ import print_function
print("hello")
print(u"hello")
print(a, file=sys.stderr)

View File

@ -1,20 +0,0 @@
#!/usr/bin/env python2
from __future__ import unicode_literals as _unicode_literals
from __future__ import absolute_import
from __future__ import print_function as lol, with_function
u'hello'
U"hello"
Ur"hello"
# output
#!/usr/bin/env python2
from __future__ import unicode_literals as _unicode_literals
from __future__ import absolute_import
from __future__ import print_function as lol, with_function
"hello"
"hello"
r"hello"

View File

@ -724,24 +724,15 @@ def test_lib2to3_parse(self) -> None:
straddling = "x + y" straddling = "x + y"
black.lib2to3_parse(straddling) black.lib2to3_parse(straddling)
black.lib2to3_parse(straddling, {TargetVersion.PY27})
black.lib2to3_parse(straddling, {TargetVersion.PY36}) black.lib2to3_parse(straddling, {TargetVersion.PY36})
black.lib2to3_parse(straddling, {TargetVersion.PY27, TargetVersion.PY36})
py2_only = "print x" py2_only = "print x"
black.lib2to3_parse(py2_only)
black.lib2to3_parse(py2_only, {TargetVersion.PY27})
with self.assertRaises(black.InvalidInput): with self.assertRaises(black.InvalidInput):
black.lib2to3_parse(py2_only, {TargetVersion.PY36}) black.lib2to3_parse(py2_only, {TargetVersion.PY36})
with self.assertRaises(black.InvalidInput):
black.lib2to3_parse(py2_only, {TargetVersion.PY27, TargetVersion.PY36})
py3_only = "exec(x, end=y)" py3_only = "exec(x, end=y)"
black.lib2to3_parse(py3_only) black.lib2to3_parse(py3_only)
with self.assertRaises(black.InvalidInput):
black.lib2to3_parse(py3_only, {TargetVersion.PY27})
black.lib2to3_parse(py3_only, {TargetVersion.PY36}) black.lib2to3_parse(py3_only, {TargetVersion.PY36})
black.lib2to3_parse(py3_only, {TargetVersion.PY27, TargetVersion.PY36})
def test_get_features_used_decorator(self) -> None: def test_get_features_used_decorator(self) -> None:
# Test the feature detection of new decorator syntax # Test the feature detection of new decorator syntax
@ -1436,27 +1427,6 @@ def test_bpo_2142_workaround(self) -> None:
actual = diff_header.sub(DETERMINISTIC_HEADER, actual) actual = diff_header.sub(DETERMINISTIC_HEADER, actual)
self.assertEqual(actual, expected) self.assertEqual(actual, expected)
@pytest.mark.python2
def test_docstring_reformat_for_py27(self) -> None:
"""
Check that stripping trailing whitespace from Python 2 docstrings
doesn't trigger a "not equivalent to source" error
"""
source = (
b'def foo():\r\n """Testing\r\n Testing """\r\n print "Foo"\r\n'
)
expected = 'def foo():\n """Testing\n Testing"""\n print "Foo"\n'
result = BlackRunner().invoke(
black.main,
["-", "-q", "--target-version=py27"],
input=BytesIO(source),
)
self.assertEqual(result.exit_code, 0)
actual = result.stdout
self.assertFormatEqual(actual, expected)
@staticmethod @staticmethod
def compare_results( def compare_results(
result: click.testing.Result, expected_value: str, expected_exit_code: int result: click.testing.Result, expected_value: str, expected_exit_code: int
@ -2086,36 +2056,6 @@ def test_get_sources_with_stdin_filename_and_force_exclude(self) -> None:
) )
@pytest.mark.python2
@pytest.mark.parametrize("explicit", [True, False], ids=["explicit", "autodetection"])
def test_python_2_deprecation_with_target_version(explicit: bool) -> None:
args = [
"--config",
str(THIS_DIR / "empty.toml"),
str(DATA_DIR / "python2.py"),
"--check",
]
if explicit:
args.append("--target-version=py27")
with cache_dir():
result = BlackRunner().invoke(black.main, args)
assert "DEPRECATION: Python 2 support will be removed" in result.stderr
@pytest.mark.python2
def test_python_2_deprecation_autodetection_extended() -> None:
# this test has a similar construction to test_get_features_used_decorator
python2, non_python2 = read_data("python2_detection")
for python2_case in python2.split("###"):
node = black.lib2to3_parse(python2_case)
assert black.detect_target_versions(node) == {TargetVersion.PY27}, python2_case
for non_python2_case in non_python2.split("###"):
node = black.lib2to3_parse(non_python2_case)
assert black.detect_target_versions(node) != {
TargetVersion.PY27
}, non_python2_case
try: try:
with open(black.__file__, "r", encoding="utf-8") as _bf: with open(black.__file__, "r", encoding="utf-8") as _bf:
black_source_lines = _bf.readlines() black_source_lines = _bf.readlines()

View File

@ -77,6 +77,9 @@ async def check(header_value: str, expected_status: int = 400) -> None:
await check("ruby3.5") await check("ruby3.5")
await check("pyi3.6") await check("pyi3.6")
await check("py1.5") await check("py1.5")
await check("2")
await check("2.7")
await check("py2.7")
await check("2.8") await check("2.8")
await check("py2.8") await check("py2.8")
await check("3.0") await check("3.0")
@ -137,10 +140,6 @@ async def check(header_value: str, expected_status: int) -> None:
await check("py36,py37", 200) await check("py36,py37", 200)
await check("36", 200) await check("36", 200)
await check("3.6.4", 200) await check("3.6.4", 200)
await check("2", 204)
await check("2.7", 204)
await check("py2.7", 204)
await check("3.4", 204) await check("3.4", 204)
await check("py3.4", 204) await check("py3.4", 204)
await check("py34,py36", 204) await check("py34,py36", 204)

View File

@ -55,12 +55,6 @@
"tupleassign", "tupleassign",
] ]
SIMPLE_CASES_PY2 = [
"numeric_literals_py2",
"python2",
"python2_unicode_literals",
]
EXPERIMENTAL_STRING_PROCESSING_CASES = [ EXPERIMENTAL_STRING_PROCESSING_CASES = [
"cantfit", "cantfit",
"comments7", "comments7",
@ -134,12 +128,6 @@ def check_file(filename: str, mode: black.Mode, *, data: bool = True) -> None:
assert_format(source, expected, mode, fast=False) assert_format(source, expected, mode, fast=False)
@pytest.mark.parametrize("filename", SIMPLE_CASES_PY2)
@pytest.mark.python2
def test_simple_format_py2(filename: str) -> None:
check_file(filename, DEFAULT_MODE)
@pytest.mark.parametrize("filename", SIMPLE_CASES) @pytest.mark.parametrize("filename", SIMPLE_CASES)
def test_simple_format(filename: str) -> None: def test_simple_format(filename: str) -> None:
check_file(filename, DEFAULT_MODE) check_file(filename, DEFAULT_MODE)
@ -219,6 +207,12 @@ def test_patma_hint() -> None:
exc_info.match(black.parsing.PY310_HINT) exc_info.match(black.parsing.PY310_HINT)
def test_python_2_hint() -> None:
with pytest.raises(black.parsing.InvalidInput) as exc_info:
assert_format("print 'daylily'", "print 'daylily'")
exc_info.match(black.parsing.PY2_HINT)
def test_docstring_no_string_normalization() -> None: def test_docstring_no_string_normalization() -> None:
"""Like test_docstring but with string normalization off.""" """Like test_docstring but with string normalization off."""
source, expected = read_data("docstring_no_string_normalization") source, expected = read_data("docstring_no_string_normalization")
@ -245,13 +239,6 @@ def test_numeric_literals_ignoring_underscores() -> None:
assert_format(source, expected, mode) assert_format(source, expected, mode)
@pytest.mark.python2
def test_python2_print_function() -> None:
source, expected = read_data("python2_print_function")
mode = replace(DEFAULT_MODE, target_versions={black.TargetVersion.PY27})
assert_format(source, expected, mode)
def test_stub() -> None: def test_stub() -> None:
mode = replace(DEFAULT_MODE, is_pyi=True) mode = replace(DEFAULT_MODE, is_pyi=True)
source, expected = read_data("stub.pyi") source, expected = read_data("stub.pyi")

12
tox.ini
View File

@ -5,7 +5,7 @@ envlist = {,ci-}py{36,37,38,39,310,py3},fuzz
setenv = PYTHONPATH = {toxinidir}/src setenv = PYTHONPATH = {toxinidir}/src
skip_install = True skip_install = True
# We use `recreate=True` because otherwise, on the second run of `tox -e py`, # We use `recreate=True` because otherwise, on the second run of `tox -e py`,
# the `no_python2` tests would run with the Python2 extra dependencies installed. # the `no_jupyter` tests would run with the jupyter extra dependencies installed.
# See https://github.com/psf/black/issues/2367. # See https://github.com/psf/black/issues/2367.
recreate = True recreate = True
deps = deps =
@ -15,15 +15,9 @@ deps =
commands = commands =
pip install -e .[d] pip install -e .[d]
coverage erase coverage erase
pytest tests --run-optional no_python2 \ pytest tests --run-optional no_jupyter \
--run-optional no_jupyter \
!ci: --numprocesses auto \ !ci: --numprocesses auto \
--cov {posargs} --cov {posargs}
pip install -e .[d,python2]
pytest tests --run-optional python2 \
--run-optional no_jupyter \
!ci: --numprocesses auto \
--cov --cov-append {posargs}
pip install -e .[jupyter] pip install -e .[jupyter]
pytest tests --run-optional jupyter \ pytest tests --run-optional jupyter \
-m jupyter \ -m jupyter \
@ -43,7 +37,7 @@ deps =
commands = commands =
pip install -e .[d] pip install -e .[d]
coverage erase coverage erase
pytest tests --run-optional no_python2 \ pytest tests \
--run-optional no_jupyter \ --run-optional no_jupyter \
!ci: --numprocesses auto \ !ci: --numprocesses auto \
ci: --numprocesses 1 \ ci: --numprocesses 1 \