Conversation
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
|
If this PR is merged, it looks to me like #146402 can (and should) be reverted. |
skirpichev
left a comment
There was a problem hiding this comment.
No, I doubt that this is a right approach. We must handle error in same way for all libm functions.
Perhaps, m_atan2() (like m_log1p() we have currently) could be restored (see #122681) to workaround broken platform functions.
The mathmodule currently has 3 ways of handling 1-argument libm functions:
Maybe it's not so bad to have |
|
Perhaps, we could modify math_2() helper to check that errno!=EDOM, if inputs and output are finite. When domain error occurs - result should be nan. But I think that a little wrapper for libm's atan2 is better, if we are going to add some workaround for the given issue. |
skirpichev
left a comment
There was a problem hiding this comment.
Please revert unrelated changes.
|
@vstinner, can you look at this at your convenience? |
| errno = 0; | ||
| phi = atan2(z.imag, z.real); /* should not cause any exception */ | ||
| if (errno != 0) | ||
| return math_error(); |
There was a problem hiding this comment.
I expected a test_cmath failure when this code path is removed. Is it because glibc math library doesn't errno in this case?
There was a problem hiding this comment.
Yes. With the exception of the Intel and Solaris math libraries, errno is not set in this case by any math library that Python cares about. I say this because test_phase in Lib/test/test_cmath.py has asserted that return values are correct since Python 3.14, and nobody has complained. If errno were set by the C math library, the unittest would fail with ValueError: math domain error.
🌱 Some math libraries (e.g., musl) don't set errno for anything, so Python cannot rely on errno for detecting overflow or invalid. I would think that errno checking can be removed everywhere....
|
You also could use new helper function for copysign. On another hand, I would prefer just remove errno stuff from math_2(), per #156145. It should be possible for all two-argument functions: atan2, atan2pi, copysign, remainder and fmod (which could utilize same wrapper). |
My thinking is only to do what is necessary to fix the bug (the incorrect results when building with Intel or Solaris math libraries). Then, I would recommend applying this fix to Python 3.14 and 3.15. With that in mind, I don't want to make any enhancements that are not necessary to fix the reported bug.
Yes, 156145 can have a more ambitious goal. Can you trigger the buildbots to run this PR? I am not able to check myself that it works on Solaris. (I do think it will work based on what I read in the Solaris bug report.) |
Then we loose chance of using new helper function for copysign().
!buildbot Solaris |
|
!buildbot Solaris |
|
🤖 New build scheduled with the buildbot fleet by @skirpichev for commit 5b6d005 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F153148%2Fmerge The command will test the builders whose names match following regular expression: The builders matched are:
|
OK, I'll add that commit tomorrow. Since it will be a separate commit, it can easily be reverted (or skipped when squashing all the commits before the final merge). The same is true for the comments Victor requested--they're in their own separate commit. |
|
I'm happy to see that the math and cmath tests pass on Solaris. |
|
Hi @StanFromIreland, Can you please advise whether I should create a separate bug report for an issue @serhiy-storchaka observed: or whether #153144 can be updated with this extra information? Note that #153144 affects the current version of the Intel math library and the Solaris math library, so it's correctly labelled |
|
Hum. It's uneasy for me to take a decision on this change. The bug report is about supporting Intel compiler (icx) which sets errno for atan2() and phase(). But the PR also changes cmath.phase() behavior on overflow and underflow of imag/real. I'm not sure if |
Python already returns the correct result, This PR changes cmath.phase() behavior on UNDERflow of imag/real (bug observed in #155527 (comment)). The C standard in 7.3.9.1 says:
Note that there's no possibility of overflow--the result is bounded. (The implementation may or may not actually perform a floating-point division of imag/real, but that's an implementation detail, and whether or not some intermediate calculation overflows must not affect what the user observes.) There is the possibility of underflow. The infinitely precise result of The result is too small to be represented in double precision, so it underflows to Python does not raise an exception on underflow, only on overflow. For example: I created the bug report because existing tests were failing when building Python with the Intel math library. I made this PR to ignore errno because I could see no reason for checking it, and I believed that not checking it was always correct. I did not stop to consider that the old code (checking errno) led to more Python bugs. After reading Serhiy's comment, I added tests to this PR and updated the NEWS. |
|
@serhiy-storchaka @skirpichev @picnixz: What is your opinion on changing |
|
FWIW, I don't think there is a change in behavior: atan2() shouldn't overflow. Underflows in Python are silent. |
|
I totally forgot about this PR. Let me read the issue and the history first (and now my speech resembles that of an AI...) |
Well, fixing a bug is a change in behavior. But I know what you mean. Let me walk through an example (using Python 3.13.5 built to use GNU libm). and note that, for both of them, imag/real underflows to zero: The math module's In the first case, the result underflows to DetailsThe wrapper In the second case, the infinitely precise result is very close to Pi, and libm rounds the result to three point something in order to return a double precision result. The result did not underflow to zero (nor did it overflow to infinity), so GNU's libm leaves errno unchanged. Python code had set Now, we get to the bug. Python calls GNU libm's In the second case, GNU's With this PR: |
|
@picnixz: So did you find time to review this change? |
|
No I'm sorry. I am quite busy in my daily job and haven't found time for that. I think I will have time on Sunday as I'm flying (or next thursday as I'm flying back). But the past weeks have been very busy (I can only do simple triaging or review). Now, after a quick glance:
This makes at least Did I catch the issue well? we wouldn't break code though, just fixing things. What I'm worried about however is implementations that really want to raise an EDOM for
In addition, Annex F says what to return when using (0,0). I would have imagined:
But in the second case, we can say it's garbage in-garbage out and it's likely an unsupported compiler (even less supported than icx!) So... I guess we canj ust ignore EDOM? Did I get it right? |
Done.
Yes. Although, technically, it's about the C math library, not the compiler. (It's possible to mix and match, e.g., use the Intel or Solaris math libraries with GCC.) Perhaps the title of my bug report for icx was too informal. Note that a math library might not strictly conform to Annex F for many reasons. Here, we only care about the values it returns for atan2(), which I would venture to guess would be correct even if the compiler and math library as a whole cannot claim to be compliant with the entirety of Annex F. Finally, I would like to suggest that, in my opinion, raising an exception where one should not happen is a serious bug. A Python programmer would not be expected to put My understanding is that bugfixes for Python 3.13 must be merged this month. |
|
Do not click the "Update branch" button without a good reason because it notifies everyone watching the PR that there are new changes, when there are not, and it uses up limited CI resources. |
|
We unfortunately get past 3.13 release and it is not security-only. I am ok with putting it in 3.16 and maybe 3.14/3.15 but I would be interested in having some NumPy's opinion just in case (cc @ngoldbaum). I think fixing the phase/atan2 inconsistency takes precedence over other behavioral change concerns. |
|
Thanks for the ping. NumPy already behaves the way CPython will behave after this PR. The only difference is that NumPy can optionally surface the underflow, and it reports it as an underflow, never as an overflow. |
|
Yeah so this matches what users will expect. I am fine with this PR then! thanks for doing it. |
| @@ -0,0 +1,7 @@ | |||
| Given a complex number ``z`` for which ``z.imag/z.real`` underflows to zero, | |||
There was a problem hiding this comment.
This NEWS entry is waaaay too long IMO. Can we make it a bit shorter?
| else | ||
| return PyFloat_FromDouble(phi); | ||
| phi = atan2(z.imag, z.real); | ||
| /* gh-153144: Ignore atan2() errno on purpose since it can optionally be |
There was a problem hiding this comment.
Question: should we clear errno here though? someone calling this function from C may want to know whether errno is appropriately set (or ignored). I would be willing to clear it before calling atan2() but I am not sure it is actually better.
| { | ||
| double x, y, r; | ||
| if (!_PyArg_CheckPositional(funcname, nargs, 2, 2)) | ||
| return NULL; |
There was a problem hiding this comment.
Please add braces around the return
There was a problem hiding this comment.
This was copied from line 1034 above. Should I add braces to that as well? (That would be my preference.)
| } | ||
|
|
||
| FUNC2(copysign, copysign, | ||
| FUNC2NE(copysign, copysign, |
There was a problem hiding this comment.
Oh so copysign is also altered? can you explain why here again? is it the same reason? I would appreciate a separate NEWS entry for that as well.
There was a problem hiding this comment.
I did this at Sergey's request.
Of course, I agree it's the right thing to do, or I would not have done it. The comment for math_2ne() is:
/* variant of math_2, to be used when the function being wrapped is known NOT
to need error checking (i.e., no overflow, invalid, or divide-by-zero). */
which certainly applies to copysign. Copying the sign bit cannot possibly cause overflow, invalid, or divide-by-zero. Now that the wrapper math_2ne() exists, it absolutely should be used for copysign. I expect it has a significant performance benefit, and it's also about having a sense of craftsmanship and taking pride in one's work and in the Python code. Given the existence of the wrapper math_2ne(), it would be confusing for somebody looking at copysign to see that math_2() is used.
If you feel copysign deserves its own NEWS entry, maybe it's better to revert the change here and create a separate PR for it? Please let me know what you think, and in particular, if you're inclined to merge a separate PR. If nobody is going to merge it, I don't want to waste my time creating it. Sergey's warning was if I don't add this code to this PR, we lose chance of using new helper function for copysign(). I don't want to lose the chance. I'm also perfectly OK with keeping the code change here in this PR and making a separate NEWS entry. Please advise.
The C23 standard states that for
atan2andatan2pi:Since Python should not raise ValueError in either of these cases (i.e., when both arguments are zero or when the computation underflows), this PR avoids checking
errnowhen calling these trig functions. As a bonus,math.atan2()is about 4% faster.math.atan2(0.0, 0.0)andcmath.phase(0.0)using icx #153144The statement about range error in the standard should, I think, be interpreted as: