Warm tip: This article is reproduced from serverfault.com, please click

how can I asynchronously map/filter an asynchronous iterable?

发布于 2016-03-18 20:40:08

Let's say I have an asynchronous iterable that I can pass over using async for, how then can I then map and filter it to a new asynchronous iterator? The following code which is an adaptation of how I'd do the same thing with a synchronous iterable doesn't work, since yield isn't allowed inside async defs.

async def mapfilter(aiterable, p, func):
    async for payload in aiterable:
        if p(payload):

            # This part isn't allowed, but hopefully it should be clear
            # what I'm trying to accomplish.
            yield func(payload)
Questioner
Te-jé Rodgers
Viewed
0
lbonn 2016-11-07 22:49:40

A recently published PEP draft (PEP 525), whose support is scheduled for Python 3.6, proposes to allow Asynchronous Generators with the same syntax you came up with.

Meanwhile, you can also use the asyncio_extras library mentioned by CryingCyclops in its comment if you don't want to deal with the asynchronous iterator boilerplate.

From the docs:

@async_generator
async def mygenerator(websites):
    for website in websites:
        page = await http_fetch(website)
        await yield_async(page)

async def fetch_pages():
    websites = ('http://foo.bar', 'http://example.org')
    async for sanitized_page in mygenerator(websites):
        print(sanitized_page)

There is also the async_generator library which supports yield from constructs.