Inconsistencies and special treatment of %e vs. %pi (and numer, %enumer mess)

Dear all,

First, please excuse the rather long mail. If it's too long for you to read
it completely, please read only the example codes that show the problem,
and then skip to "TLDR / Way forward?".

While digging into the simplifier and evaluator, I noticed that %e receives
an almost ridiculous amount of special treatment compared to, e.g., %pi.

   - %e is the *only atom* that can be simplified to something different by
   the simplifier: If the global flags numer and %enumer are both true, %e
   is simplified to its numerical value. It's likely that there is code at
   other places that (wrongly, but understandably) assumes that atoms are
   inherently simplified and don't need to go through the simplifier, which
   would cause bugs. The other constants like %pi aren't handled by the
   simplifier.
   - There is some really ugly hackery going on in the evaluator that
   explicitly targets %e in conjunction with numer/%enumer.

The global flag %enumer = false (default) is supposed to "protect" %e from
becoming a float when numer is true in cases of %e^x, where x is symbolic.
At first glance, this works:

(%i1) %e^x + sin(%e), numer;
(%o1) %e^x + 0.41078129050290885

Here, %e^x is correctly left alone, while sin(%e) becomes a float.
The following also works:

(%i2) sin(%e^x), numer;
(%o2) sin(%e^x)

But now let's resimplify that with numer on (classically, this would be
done with expand(%, 0, 0), but we have resimplify now):

(%i3) resimplify(%), numer;
(%o3) sin(2.718281828459045^x)

Oops! Now %e became a float, even though it's the base with a symbolic
exponent, which is exactly the case where it should have been "protected"
by %enumer = false.

What's going on here?

   - When typing just sin(%e^x), numer in the REPL, the evaluator starts by
   simplifying the leaves %e and x, then builds and simplifies %e^x, and
   finally builds and simplifies sin(%e^x). It works its way from the
   leaves to the root, *bottom-up*.
   - But when we resimplify, all simp tags are stripped from the entire
   expression, and the simplifier is called on the root, working in a
   *top-down* fashion. The sin simplifier calls simpcheck on its argument,
   which, due to a momentous commit described here
   <https://sourceforge.net/p/maxima/bugs/123/>, dynamically binds %enumer
   to numer (true). This "infects" the entire sub-tree, and all instances
   of %e inside it are converted to a float by the simplifier, no matter
   what.

So we get a different outcome depending on whether we work bottom-up or
top-down. It doesn't end there, though. Here's an example that shows how %e
and %pi are treated in a fundamentally different way:

/* OK, both become floats: */
(%i4) %pi*y + %e*x, numer;
(%o4) 3.141592653589793*y + 2.718281828459045*x

/* Not OK, %e stays symbolic: */
(%i5) [%pi, %e], numer;
(%o5) [3.141592653589793, %e]

/* If we write -%e instead of %e, it becomes a float: */
(%i6) [%pi, -%e], numer;
(%o6) [3.141592653589793, -2.718281828459045]

*TLDR / Way forward?*

   - In summary, the handling of %e, numer and %enumer is an inconsistent
   mess that needs lots of special cases in the code, and it doesn't even work
   as intended.
   - I think that, as much as possible, this extra treatment of %e should
   go away. It would make the code simpler and the results more consistent.
   - The idea behind %enumer = false isn't bad, but there's a fundamental
   problem with the way it's supposed to work: When working bottom-up, we
   first see %e, but *we don't have the context*, i.e. we don't know where
   that %e sits inside a parent expression (if any).
      - Is it the %e in %e^x? Then it must not become a float.
      - Is it any %e in %e^%e? Then it must become a float.
      - Is it the %e in sin(%e)? Then it must become a float.
      - Is it just a standalone %e? Then it must become a float.
   - There's one way it could work, although it comes with a catch:
      - We stop handling %e in simplifya entirely. *Nice side effect: Atoms
      are always simplified!*
      - Instead, we treat %e like any other constant, e.g. %pi, and let the
      evaluator convert it to float unconditionally when numer is true.
      - Afterwards, if %enumer is false, we look explicitly for
      2.718281828459045^(symbolic) and convert that back to %e^(symbolic).
      Note that this is an exact comparison to %e-val that will not be
      affected by floating point issues.
      - The catch: If the user manually enters 2.718281828459045^x, it will
      become %e^x. Maybe that's acceptable? I don't see any other way of
      achieving the desired behavior.
   - Alternatively, we could degrade %enumer to a selective switch just for
   %e. With %enumer = false, it would always prevent %e from becoming a
   float when numer is true. But that would include things like sin(%e),
   where the user would probably expect a float. This behavior would probably
   not be very useful, and I don't like the idea of changing the meaning of a
   flag in a significant way, as it would destroy backwards-compatibility.

Looking forward to the community's opinions on this.

Best regards
David Scherfgen

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss