Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Document execution order #888
Conversation
|
Why document any one thing in particular? The order is universally "postorder" (on the AST), i.e. "more nested" expressions first (effectively mandated by "expressions depend on the result of their subexpressions"), and between siblings it's specifically left-to-right (outside of perhaps assignments' LHS? I forget how we resolved that one). Can't we specify it in one place? Doing it on a case-by-case basis seems inefficient and would suggest the order isn't uniformly defined, whereas it is (otherwise e.g. the borrow-checker couldn't possibly work). |
|
I thought about that after I went to bed as well. I could add an evaluation order section to |
|
I've completely rewritten this PR to now document the default at the top level. I'll be doing the tuple changes as their own PR. |
|
I don't think I know enough to review this. Is this an actual commitment the language can make? Does I would expect this to have some more precise language. Perhaps make it clear that this left-to-right ordering is referring to the operands of an expression, and this is not related operator precedence and associativity? (Or, at least, spell out the relationship with operator precedence and associativity?) I think "operands" is a little more specific than "subexpressions" or "inner expressions". I'm also a little unclear on how this relates to execution order, which is sometimes defined relative to "sequence" or "critical execution" points. |
Havvy
changed the title
|
For reordering expressions: If there are no observable side effects done by doing so, then expressions can be reordered in their evaluation order. Heck, they can even be evaluated at compile time under certain conditions. But this is also true about the ordering of statements in a block expression. And yet, those aren't semantics of the language. They're optimizations in the implementation. Anywhere there are actual side effects, this becomes true. With respect to "operands", I think "subexpressions" makes a more clear term. Most expressions contain inner expressions that are subsumed by that expression. Operand would have to be defined and outside of arithemetic operator expressions, you don't see the term used. With respect to "operator precedence", I should add a note about that. Evaluation order and execution order are basically synonymous to me. I don't think I've seen a distinction between the two terms in any literature I've read. |
|
The reason I suggest "operand" is because for me, "sub-expression" can be ambiguous if it is recursive. Like, the "borrow expression" clearly (to me) has one operand, but has a limitless number of sub-expressions if you consider it recursively. Also, "operand" is already defined and used throughout I guess I just feel a little uneasy to have an hard rule of "left to right", when that's not universally true if optimizers are allowed to reorder based on some unwritten rules. Like C++17 very clearly defines ordering with respect to value computation and side effects. And Java very clearly expresses the order and caveats on when it can be rewritten. Many other specs just vaguely mention "left-to-right" (like Go and Javascript and D and C#), so it is not unusual, but it seems strange to me to be so imprecise in a spec. I think it is interesting that C# used to have an ordering based on well-defined side effects, but that seems to have been removed and it only mentions left-to-right ordering now. |
I've italicized "operand" showing that it is a definition. I didn't actually remember that being added to the reference, so I basically tried to redefine it in the previous commit. I don't like the term, but since it's already there, I'll just use it. I also put in a note saying that operator precedence determines the operands of an expression. That section could probably be written in a style that better expresses that perspective, but I'm trying to keep this change minimal. I also stated that the evaluation of operands is done prior to applying the effect. This goes in line with the beginning of the chapter with what the meaning of an expression. Note also that this only describes the default. Expressions that deviate from the default already should describe their evaluation order.
|
See the commit message. Since operand is used already, I'll continue to use it. Didn't actually see it before. What optimizer do is something that we don't concern ourselves with in the reference. It can do much worse things to the semantics of the language already by eliding copies and movies and function calls or replacing function calls with a constant instead. Yes, it can reorder if it doesn't change the ultimate effect. Nothing in the reference is universal in the face of an optimizer. |
|
@ehuss Do you still feel unqualified to review this? |
|
I would be more comfortable if someone from the language team reviewed this. It looks good to me, and it does follow the convention used in several other languages as noted above. I'm just uncertain if that's the most appropriate way to define the order. Otherwise I'm fine with it. |
It is universally true though. Optimizations are only allowed to reorder things in a way that causes no change of observable behavior (the "as-if" rule). The rules for what optimizations are allowed to do are not unwritten, they are "you must only do things where the resulting program only behaves in ways that the original program could already behave as" -- this is the universal correctness condition for all compilers of all programming languages. That's just like how the spec says that It is the responsibility of optimizations to justify their correctness against the spec, and not vice versa. |
That would be horrible! Everything in the reference is universal for all correct optimizers, when considering the observable effects of running a Rust program. Don't think in terms of assembly, think of the resulting program as a black box that you can only probe by running it, feeding it input and observing its output. Everything that happens that way must conform to what the reference says. The reference says nothing about the inside of that black box, but that does not at all mean that the reference is any less universal. The inside of that black box is simply out of scope, just like, e.g., the exact formatting of error messages. |
| @@ -45,13 +45,12 @@ | |||
| An expression may have two roles: it always produces a *value*, and it may have | |||
| *effects* (otherwise known as "side effects"). An expression *evaluates to* a | |||
| value, and has effects during *evaluation*. Many expressions contain | |||
| sub-expressions (operands). The meaning of each kind of expression dictates | |||
joshtriplett
Nov 9, 2020
Member
This is a nit, and not a semantic issue with the description, but: I have a moderate preference that we continue to use "sub-expressions" in general, and only use "operands" for "sub-expressions of an operator".
This is a nit, and not a semantic issue with the description, but: I have a moderate preference that we continue to use "sub-expressions" in general, and only use "operands" for "sub-expressions of an operator".
joshtriplett
Nov 11, 2020
Member
Withdrawing this comment; the only case that really bugs me is the idea that in f(a, b), f, a, and b would all be considered "operands" of the function call expression, and I can live with that terminology in the interests of precision.
Withdrawing this comment; the only case that really bugs me is the idea that in f(a, b), f, a, and b would all be considered "operands" of the function call expression, and I can live with that terminology in the interests of precision.
Havvy
Nov 12, 2020
Author
Collaborator
FWIW, I didn't like the word beforehand, but after this and the tuple PR, I've found it works well. The only alternative I can think of would be to state that when we talk about sub-expressions, we only mean the top-most level, not the subexpressions of the subexpressions. And requiring global context seems like a bad idea.
FWIW, I didn't like the word beforehand, but after this and the tuple PR, I've found it works well. The only alternative I can think of would be to state that when we talk about sub-expressions, we only mean the top-most level, not the subexpressions of the subexpressions. And requiring global context seems like a bad idea.
| @@ -85,6 +84,33 @@ in the order given by their associativity. | |||
| | `=` `+=` `-=` `*=` `/=` `%=` <br> `&=` <code>|=</code> `^=` `<<=` `>>=` | right to left | | |||
| | `return` `break` closures | | | |||
|
|
|||
| ## Evaluation order of operands | |||
|
|
|||
| Unless otherwise stated on the expression's page, evaluation of the operands of | |||
joshtriplett
Nov 9, 2020
Member
Can we cross-reference any cases where this isn't true from within this section, to not leave people wondering "what expressions don't evaluate their sub-expressions left-to-right"?
Can we cross-reference any cases where this isn't true from within this section, to not leave people wondering "what expressions don't evaluate their sub-expressions left-to-right"?
Havvy
Nov 12, 2020
Author
Collaborator
I listed every case where it is true. I was being lazy with that sentence prior. I'm still being lazy by not actually linking to those expression pages, but that can always be done later and they're in the sidebar.
I listed every case where it is true. I was being lazy with that sentence prior. I'm still being lazy by not actually linking to those expression pages, but that can always be done later and they're in the sidebar.
nikomatsakis
Nov 19, 2020
Contributor
One prominent case that is not mentioned here is a = b. Our execution odrer there, I believe, is to evaluate b before a, though I want to double check now =)
One prominent case that is not mentioned here is a = b. Our execution odrer there, I believe, is to evaluate b before a, though I want to double check now =)
RalfJung
Nov 19, 2020
Member
Hm, I seem to recall @eddyb saying that it actually is even more complicated than that, but I forgot the details...
Hm, I seem to recall @eddyb saying that it actually is even more complicated than that, but I forgot the details...
|
A couple of minor nits, but otherwise, this looks great. With my lang team hat on (though not speaking for the rest of the lang team), |
Borrowing language from Ralf Jung from a comment on PR 888 (rust-lang#888 (comment)) describe the relationship of optimizers and what the reference states.
Borrowing language from Ralf Jung from a comment on PR 888 (rust-lang#888 (comment)) describe the relationship of optimizers and what the reference states.
|
|
||
| > **Note**: Since this is applied recursively, expressions are also evaluated | ||
| > from innermost to outermost, ignoring siblings until there are no inner | ||
| > subexpressions. |
nikomatsakis
Nov 19, 2020
Contributor
I would prefer to extend this PR with coverage of
- assignment expressions
- the order of "augmented assignment" like
+=, particularly in the face of operator overloading -- this gets a bit subtle, see e.g. rust-lang/rust#28160 -- I think at this point it's clear though that we just have to document the current behavior and live with it
How much did you test to verify the execution order for various corner cases? (e.g., overloaded index?) I think that what you wrote above is correct, but I'd love to see a comprehensive set of test cases added to the repository, e.g., in a place like src/test/ui/evaluation_order.rs =) this doesn't all have to fall on you, of course, we could just write to write up a list of those tests and see if we can advertise folks to write them up
I would prefer to extend this PR with coverage of
- assignment expressions
- the order of "augmented assignment" like
+=, particularly in the face of operator overloading -- this gets a bit subtle, see e.g. rust-lang/rust#28160 -- I think at this point it's clear though that we just have to document the current behavior and live with it
How much did you test to verify the execution order for various corner cases? (e.g., overloaded index?) I think that what you wrote above is correct, but I'd love to see a comprehensive set of test cases added to the repository, e.g., in a place like src/test/ui/evaluation_order.rs =) this doesn't all have to fall on you, of course, we could just write to write up a list of those tests and see if we can advertise folks to write them up
Havvy
Nov 20, 2020
Author
Collaborator
I haven't done any testing of the corner cases. Although I did just make one for Index. But given that for all of these operations, the actual method call happens after the operands are evaluated, I can't see execution order changing by using language known impls or trait impls.
I also am not sure a single evaluation_order.rs test file is the best place. I would think you'd want src/test/ui/exprs/%expr_kind%/evaluation_order.rs so that it's grouped with other tests of the expression kind. But of course, that'd require quite a lot of reorganization of the tests and ultimately all of this falls outside of the domain of the reference docs.
I also just did a quick test for assignment operators, and they apparently evaluate the right expression before the left expression. That I did not know, so that does require updating this PR.
I haven't done any testing of the corner cases. Although I did just make one for Index. But given that for all of these operations, the actual method call happens after the operands are evaluated, I can't see execution order changing by using language known impls or trait impls.
I also am not sure a single evaluation_order.rs test file is the best place. I would think you'd want src/test/ui/exprs/%expr_kind%/evaluation_order.rs so that it's grouped with other tests of the expression kind. But of course, that'd require quite a lot of reorganization of the tests and ultimately all of this falls outside of the domain of the reference docs.
I also just did a quick test for assignment operators, and they apparently evaluate the right expression before the left expression. That I did not know, so that does require updating this PR.
nikomatsakis
Nov 20, 2020
Contributor
@Havvy yeah I totally agree that a "directory" is the right place to sort those kinds of tests.
@Havvy yeah I totally agree that a "directory" is the right place to sort those kinds of tests.
This commit does not try to define their evaluation order, they just removed them from the list of shared LTR evaluated expressions.
I'm being wish-washy with expression/operand in the syntax section. I'm not quite sure if we should call them `place operands` everywhere.
|
I need to still do compound assignment expressions, but I'm getting tired. Since I was modifying assignment expressions, I figured I'd give it the same treatment I gave tuples (and would like to give to all expressions). Image of rendered change
|
|
@Havvy this is looking good, are you still expecting to make more additions here, or would you rather land for now? |
|
I keep putting off finishing up the section on compound assignment; lack of energy after work or last week a patch dropped in an MMO. If you want to merge as is, I can send that section as its own PR. |
|
Given that there will probably be needed discussion on compound assignment operators, I wish to land that part separately from this. |
Thank eddyb and Steve Klabnik for complaining on Twitter for this PR.
Closes #248
The order is now documented. I considered using an example over expression evaluation using println!, but the twitter thread is explicitly using iterators and this is probably the most common reason to care about evaluation order in tuple expressions anyways.
I also cleaned up the page to bring it into style compliance. I did both at the same time, so there's only one noisy commit. Sorry to the reviewer.