Issue 21964: inconsistency in list-generator comprehension with yield(-from)

Will playing with generators and `yield from` I found some inconsistency.

list comprehension with yield(-from) would return a generator.
generator comprehension would yield some None in the middle of the expected values.

Examples:
    l = ["abc", range(3)]
    g1 = [(yield from i) for i in l]
    print(g)
    <generator object <listcomp> at 0x7f5ebd58b690>
    print(list(g))
    ['a', 'b', 'c', 0, 1, 2]  # this result is super cool !

    g2 = ((yield from i) for i in l)
    print(g2)
    <generator object <genexpr> at 0x7f5ebd58b6e0>
    print(list(g2))
    ['a', 'b', 'c', None, 0, 1, 2, None]


For `g1`: it returns a generator because the listcomp contains a `yield from`.

For `g2` it append None because it yield the return value of `yield from i`.
It could be rewritten as:
    def comp(x):
        for i in x:
            yield (yield from i)

There seem to be two issues here:

> list comprehension with yield(-from) would return a generator.

This is somewhat surprising, and I'm not sure it's expected/documented.
The example you provided seems to behave reasonably, so I don't think we should change the behavior (unless there is some actual bug in similar example).  If anything, we could document this, but I'm not sure if it's worth doing it and where could be added.

> generator comprehension would yield some None in the middle
> of the expected values.

This also seems expected, and the behavior is consistent with the equivalent generator function.

It's a side effect of the hidden closure that provides the new scope for the iteration variable - that's an ordinary function object, so using yield or yield from turns it into a generator expression instead. Generator expressions are already generators, so using yield or yield from just adds more yield points beyond the implied ones.

I've never figured out a good way to document it - it's a natural consequence of the comprehension's closure. An explicit mention in the reference docs for comprehensions may be worth adding.

I found something else, I think it worth mentioning it.

This side-effect allows to create generators that return other values that None. And the CPython's behavior is not the same for all versions:

    >>> {(yield i) : i for i in range(2)}
    <generator object <dictcomp> at 0x7f0b98f41410>
    >>> list(_)
    # python3.(3,4,5)
    [0, 1]
    # python3.2
    [0, 1, {None: 1}] #  python3.2 appends the generator's return value.

> For `g1`: it returns a generator because the listcomp contains a `yield from`.

IMO it's a bug: [... for ... in ...] must create a list.

I think this issue was resolved by now. This is what happens on 3.11:

>>> l = ["abc", range(3)]
>>> g = [(yield from i) for i in l]
  File "<stdin>", line 1
SyntaxError: 'yield' inside list comprehension

>>> g2 = ((yield from i) for i in l)
  File "<stdin>", line 1
SyntaxError: 'yield' inside generator expression

Resolved in issue10544.

superseder: yield expression inside generator expression does nothing

nosy: + serhiy.storchaka
messages: + msg408294
resolution: out of date -> duplicate
stage: test needed -> resolved

nosy: + iritkatriel
messages: + msg408251

resolution: out of date

messages: + msg222877
versions: + Python 3.4