我在围绕Python 3的Asyncio库取得进展方面遇到了麻烦。我有一个邮政编码列表,我正在尝试对API进行异步调用,以获取每个邮政编码对应的城市和州。我可以使用for循环依次成功完成此操作,但在邮政编码列表较大的情况下,我想使其更快。
这是我的作品的一个例子
import urllib.request, json
zips = ['90210', '60647']
def get_cities(zipcodes):
zip_cities = dict()
for idx, zipcode in enumerate(zipcodes):
url = 'http://maps.googleapis.com/maps/api/geocode/json?address='+zipcode+'&sensor=true'
response = urllib.request.urlopen(url)
string = response.read().decode('utf-8')
data = json.loads(string)
city = data['results'][0]['address_components'][1]['long_name']
state = data['results'][0]['address_components'][3]['long_name']
zip_cities.update({idx: [zipcode, city, state]})
return zip_cities
results = get_cities(zips)
print(results)
# returns {0: ['90210', 'Beverly Hills', 'California'],
# 1: ['60647', 'Chicago', 'Illinois']}
这是我尝试使其异步的可怕的非功能性尝试
import asyncio
import urllib.request, json
zips = ['90210', '60647']
zip_cities = dict()
@asyncio.coroutine
def get_cities(zipcodes):
url = 'http://maps.googleapis.com/maps/api/geocode/json?address='+zipcode+'&sensor=true'
response = urllib.request.urlopen(url)
string = response.read().decode('utf-8')
data = json.loads(string)
city = data['results'][0]['address_components'][1]['long_name']
state = data['results'][0]['address_components'][3]['long_name']
zip_cities.update({idx: [zipcode, city, state]})
loop = asyncio.get_event_loop()
loop.run_until_complete([get_cities(zip) for zip in zips])
loop.close()
print(zip_cities) # doesnt work
任何帮助深表感谢。我在网上遇到的所有教程似乎都令我有些头疼。
注意:我已经看到了一些使用示例aiohttp
。我希望尽可能使用本地Python 3库。
如果您用于urllib
执行HTTP请求,那么您将无法获得任何并发性,因为它是一个同步库。包装的功能调用到urllib
在coroutine
不改变。您必须使用集成到中的异步HTTP客户端asyncio
,例如aiohttp
:
import asyncio
import json
import aiohttp
zips = ['90210', '60647']
zip_cities = dict()
@asyncio.coroutine
def get_cities(zipcode,idx):
url = 'https://maps.googleapis.com/maps/api/geocode/json?key=abcdfg&address='+zipcode+'&sensor=true'
response = yield from aiohttp.request('get', url)
string = (yield from response.read()).decode('utf-8')
data = json.loads(string)
print(data)
city = data['results'][0]['address_components'][1]['long_name']
state = data['results'][0]['address_components'][3]['long_name']
zip_cities.update({idx: [zipcode, city, state]})
if __name__ == "__main__":
loop = asyncio.get_event_loop()
tasks = [asyncio.async(get_cities(z, i)) for i, z in enumerate(zips)]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
print(zip_cities)
我知道您更喜欢只使用stdlib,但是该asyncio
库不包含HTTP客户端,因此您必须从根本上重新实现aiohttp
其一部分才能重新创建其提供的功能。我想另一种选择是urllib
在后台线程中进行调用,以使它们不会阻塞事件循环,但是在aiohttp
可用时这样做是很愚蠢的(并且首先打败了使用目的asyncio
) :
import asyncio
import json
import urllib.request
from concurrent.futures import ThreadPoolExecutor
zips = ['90210', '60647']
zip_cities = dict()
@asyncio.coroutine
def get_cities(zipcode,idx):
url = 'https://maps.googleapis.com/maps/api/geocode/json?key=abcdfg&address='+zipcode+'&sensor=true'
response = yield from loop.run_in_executor(executor, urllib.request.urlopen, url)
string = response.read().decode('utf-8')
data = json.loads(string)
print(data)
city = data['results'][0]['address_components'][1]['long_name']
state = data['results'][0]['address_components'][3]['long_name']
zip_cities.update({idx: [zipcode, city, state]})
if __name__ == "__main__":
executor = ThreadPoolExecutor(10)
loop = asyncio.get_event_loop()
tasks = [asyncio.async(get_cities(z, i)) for i, z in enumerate(zips)]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
print(zip_cities)
本文收集自互联网,转载请注明来源。
如有侵权,请联系 [email protected] 删除。
我来说两句