在 django 中使用 gevent 异常的问题

V2EX = way to explore

V2EX 是一个关于分享和探索的地方

现在注册

已注册用户请登录

推荐学习书目

› Learn Python the Hard Way

Python Sites

› PyPI - Python Package Index

› http://diveintopython.org/toc/index.html

› Pocoo

值得关注的项目

› PyPy

› Celery

› Jinja2

› Read the Docs

› gevent

› pyenv

› virtualenv

› Stackless Python

› Beautiful Soup

› 结巴中文分词

› Green Unicorn

› Sentry

› Shovel

› Pyflakes

› pytest

Python 编程

› pep8 Checker

Styles

› PEP 8

› Google Python Style Guide

› Code Style from The Hitchhiker's Guide

这是一个创建于 2690 天前的主题，其中的信息可能已经有所发展或是发生改变。

有个即时爬虫的需求，每一次请求过来，就去网站爬取多条数据，所以用到了 gevent.spawn，代码如下

 spawns =[]
 for param in param_list:
            spawns.append(gevent.spawn(s_search, param))
 gevent.joinall(spawns)

s_search 是具体的爬取的代码，第一次请求的时候正常，第二次请求异常，第三次又正常了，就这样正常一次，错误一次。异常如下:

  File "C:\Users\duanbingjie\PycharmProjects\IpProxy\IpProxy\IPrice.py", line 107, in get_price
    gevent.joinall(spawns)
  File "C:\python27\lib\site-packages\gevent\greenlet.py", line 649, in joinall
    return wait(greenlets, timeout=timeout, count=count)
  File "C:\python27\lib\site-packages\gevent\hub.py", line 1038, in wait
    return list(iwait(objects, timeout, count))
  File "C:\python27\lib\site-packages\gevent\hub.py", line 985, in iwait
    item = waiter.get()
  File "C:\python27\lib\site-packages\gevent\hub.py", line 939, in get
    Waiter.get(self)
  File "C:\python27\lib\site-packages\gevent\hub.py", line 899, in get
    return self.hub.switch()
  File "C:\python27\lib\site-packages\gevent\hub.py", line 630, in switch
    return RawGreenlet.switch(self)
LoopExit: ('This operation would block forever', <Hub at 0x987f030 select pending=0 ref=0>)

7 条回复 • 2017-08-29 15:15:32 +08:00