Consider having int() support MINUS SIGN

[rhettinger]

The int() function supports all Nd unicode digit variants so that int('\N{digit one}') == int('\N{arabic-indic digit one}').

Likewise, the int() function supports all Zs space character variants to that int('\N{space}123') == int('\N{no-break space}123').

However, the function does not support negation variants so that int('\N{hyphen-minus}\N{digit one}') succeeds while int('\N{minus sign}\N{digit one}') fails.

I found this issue while parsing a Table of Jacobi Symbol Values. All of the −1 entries are a MINUS SIGN and DIGIT ONE. I would have expected all minus sign variants to be supported.

Linked PRs

• gh-144087: Add support for unicode MINUS SIGN in int, float and complex #144095

[johnslavik]

If we normalize MINUS SIGN to HYPHEN-MINUS in _PyUnicode_TransformDecimalAndSpaceToASCII, that will also affect float and complex. I think this is the correct and desired approach.

We can consider updating _pylong.str_to_int, but it is not used by us and maybe that one would need a more general rework (rather than just changing one specific thing to increase parity) according to this comment:

cpython/Lib/_pylong.py

Lines 402 to 404 in 8951088

	def str_to_int(s):
	"""Asymptotically fast version of decimal string to 'int' conversion."""
	# FIXME: this doesn't support the full syntax that int() supports.

We don't use this one internally. We instead sometimes use _pylong.int_from_string that has sign conversion already covered by _PyUnicode_TransformDecimalAndSpaceToASCII when called.

I believe _PyUnicode_TransformDecimalAndSpaceToASCII to be the right place to handle this.

[johnslavik]

Prior discussion: #54790 (comment)

[skirpichev]

the function does not support negation variants so that int('\N{hyphen-minus}\N{digit one}') succeeds while int('\N{minus sign}\N{digit one}') fails.

Indeed, this looks odd. The \N{plus sign} is supported.

all minus sign variants

Could you be more precise here, do we need something else?

I imagine, that many dash-like characters might erroneously be used for minus sign. E.g.:

>>> s = '\u2011\N{digit one}'; print(s)
‑1
>>> -1  # find the difference? :-)
-1
>>> int(s)
Traceback (most recent call last):
  File "<python-input-6>", line 1, in <module>
    int(s)
    ~~~^^^
ValueError: invalid literal for int() with base 10: '‑1'

[johnslavik]

@skirpichev yes, but MINUS SIGN is specifically supposed to be used as the mathematical minus. That's why it ended up on Wikipedia.

IMO, MINUS SIGN is the right additional case to support, and anything else that looks similar is okay to be unsupported. We could revisit any additional cases if someone asks for it, but my gut instinct is that it's very unlikely.

[skirpichev]

If we normalize MINUS SIGN to HYPHEN-MINUS in _PyUnicode_TransformDecimalAndSpaceToASCII, that will also affect float and complex.

That's ok, as they all support unicode digits.

But not forget about Decimal's, which don't use this helper.

[johnslavik]

But not forget about Decimal's, which don't use this helper.

Good note. For the decimal type, we'd update numeric_as_ascii.

Fractions don't support MINUS SIGN as well:

>>> fractions.Fraction("\N{MINUS SIGN}1")
Traceback (most recent call last):
  ...
ValueError: Invalid literal for Fraction: '−1'
>>> fractions.Fraction("-1")
Fraction(-1, 1)

for these we'd update fractions._RATIONAL_FORMAT.

[skirpichev]

Fractions don't support this as well

Yeah. You will need to change fractions module just like _pydecimal.

[johnslavik]

FWIW, wcstol doesn't seem to support this.

However, given prior discussions years ago, I think this is still a good idea, esp. for parsing documents that contain numbers with this sign.
To me, this is similar in nature to handling the final form of capital sigma Σ in str.lower().

[johnslavik]

This feature was requested by @ambv in 2013: https://mail.python.org/pipermail/python-ideas/2013-June/021217.html.
See also https://mail.python.org/pipermail/python-ideas/2013-June/021243.html.

The last message in the thread concluded on support for this feature request: https://mail.python.org/pipermail/python-ideas/2013-June/021273.html.

Łukasz, what was the reception of this proposal at the time? Do you know if the proposal was suddenly declined or deferred at some point, or maybe just not revisited later? What do you think about it today?

[johnslavik]

CC @sethmlarson as security expert