Python 装饰器和 functools 模块

什么是装饰器?

在 Python 语言里第一次看到装饰器不免让人想到设计模式中的装饰模式——动态地给一个对象添加一些额外的职责,就增加功能来说,装饰模式比生成子类更为灵活。
好吧,Python 中的装饰器显然和装饰模式毫无关系。那 Python 中的装饰器到底是什么呢?
简而言之,装饰器提供了一种方法,在函数和类定义语句的末尾插入自动运行代码。Python 中有两种装饰器:函数装饰器和类装饰器。

函数装饰器

简单的装饰器例子:

[code]
def decorator(F): # 装饰器函数定义
print "I’m decorator"
return F

@decorator
def foo():
print ‘Hello World!’
# 上面等价于 foo = decorator(foo)

foo()
"""
I’m decorator
Hello World!
"""

decorator(foo)() # 所以这里的输出与 foo() 相同
"""
I’m decorator
Hello World!
"""
[/code]



从上面运行后结果看出,装饰器就是一个能够返回可调用对象(函数)的可调用对象(函数)。
具有封闭作用域的装饰器

[code]
def decorator(func): # 装饰器函数
print ‘in decorator’
def wrapper(*args):
print ‘in decorator wrapper’
wrapper._calls += 1
print "calls = %d" % (wrapper._calls)
func(*args)
wrapper._calls = 0
return wrapper

@decorator
def foo(x, y):
print "x = %d, y = %d" % (x, y)

foo(1, 2) # 第一次调用
"""
in decorator
in decorator wrapper
calls = 1
x = 1, y = 2
"""

foo(2, 3) # 第二次调用
"""
in decorator wrapper
calls = 2
x = 2, y = 3
"""
[/code]

可以看出第一次调用 foo(1, 2) 时,相当于

[code]
foo = decorator(foo)
foo(1, 2)
[/code]

第二次调用 foo(2, 3) 时 foo 已经为 decorator(foo) 的返回值了
再看看一个装饰器类来实现的:

[code]
class decorator: # 一个装饰器类
def __init__(self, func):
print ‘in decorator __init__’
self.func = func
self.calls = 0
def __call__(self, *args):
print ‘in decorator __call__’
self.calls += 1
print "calls = %d" % (self.calls)
self.func(*args)

@decorator
def foo(x, y):
print "x = %d, y = %d" % (x, y)

foo(1, 2) # 第一次调用
"""
in decorator __init__
in decorator __call__
calls = 1
x = 1, y = 2
"""

foo(2, 3) # 第二次调用
"""
in decorator __call__
calls = 2
x = 2, y = 3
"""
[/code]

装饰器参数

[code]
def decorator_wrapper(a, b):
print ‘in decorator_wrapper’
print "a = %d, b = %d" % (a, b)
def decorator(func):
print ‘in decorator’
def wrapper(*args):
print ‘in wrapper’
func(*args)
return wrapper
return decorator

@decorator_wrapper(1, 2) # 这里先回执行 decorator_wrapper(1, 2), 返回 decorator 相当于 @decorator
def foo(word):
print word

foo(‘Hello World!’)
"""
in decorator_wrapper
a = 1, b = 2
in decorator
in wrapper
Hello World!
[/code]

functools 模块

functools 模块中有三个主要的函数 partial(), update_wrapper() 和 wraps(), 下面我们分别来看一下吧。
partial(func[,args][, *keywords])
看源码时发现这个函数不是用 python 写的,而是用 C 写的,但是帮助文档中给出了用 python 实现的代码,如下:

[code]
def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords):
newkeywords = keywords.copy()
newkeywords.update(fkeywords)
return func(*(args + fargs), **newkeywords)
newfunc.func = func
newfunc.args = args
newfunc.keywords = keywords
return newfunc
[/code]

OK,可能一下子没看明白,那么继续往下看,看一下是怎么用的。我们知道 python 中有个 int([x[,base]]) 函数,作用是把字符串转换为一个普通的整型。如果要把所有输入的二进制数转为整型,那么就要这样写 int(’11’, base=2)。这样写起来貌似不太方便,那么我们就能用 partial 来实现值传递一个参数就能转换二进制数转为整型的方法。

[code]
from functools import partial
int2 = partial(int, base=2)
print int2(’11’) # 3
print int2(‘101′) # 5
from functools import partial
int2 = partial(int, base=2)
print int2(’11’) # 3
print int2(‘101’) # 5
[/code]

update_wrapper(wrapper, wrapped[, assigned][, updated])
看这个函数的源代码发现,它就是把被封装的函数的 module, name, doc 和 dict 复制到封装的函数中去,源码如下,很简单的几句:

[code]
WRAPPER_ASSIGNMENTS = (‘__module__’, ‘__name__’, ‘__doc__’)
WRAPPER_UPDATES = (‘__dict__’,)
def update_wrapper(wrapper,wrapped, assigned = WRAPPER_ASSIGNMENTS,updated = WRAPPER_UPDATES):
for attr in assigned:
setattr(wrapper, attr, getattr(wrapped, attr))
for attr in updated:
getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
return wrapper
[/code]

具体如何用我们可以往下看一下。

[code]
def wraps(wrapped,assigned = WRAPPER_ASSIGNMENTS,updated = WRAPPER_UPDATES):
return partial(update_wrapper, wrapped=wrapped,assigned=assigned, updated=updated)

[/code]

好,接下来看一下是如何使用的,这才恍然大悟,一直在很多开源项目的代码中看到如下使用。

[code]
from functools import wraps
def my_decorator(f):
@wraps(f)
def wrapper(*args, **kwds):
print ‘Calling decorated function’
return f(*args, **kwds)
return wrapper

@my_decorator
def example():
"""这里是文档注释"""
print ‘Called example function’

example()

# 下面是输出
"""
Calling decorated function
Called example function
"""
print example.__name__ # ‘example’
print example.__doc__ # ‘这里是文档注释’
[/code]
其它参考链接

  • http://www.wklken.me/posts/2013/08/18/python-extra-functools.html
  • http://coolshell.cn/articles/11265.html
  • Python自动下载百度音乐分类

    [python]
    #-*- coding: utf-8 -*-
    import urllib
    import re
    from Api_tieba import tieba
    class BaiduMusic:
    def __init__(self, key_words, rate=0, start_num=0, end_num=1000):
    if rate>0:
    self.tieba=tieba()
    self.tieba.login_init("xx", "xx")
    self.tieba.login()
    self.search_url="http://music.baidu.com/search/tag?key="+str(key_words)
    self.base_url="http://music.baidu.com"
    self.start_num=start_num
    self.end_num=end_num
    self.rate=rate
    self.run()
    def run(self):
    for i in range(self.start_num/20, self.end_num/20):
    url=self.search_url+"&start="+str(i*20)+"&size=20"
    html = urllib.urlopen(url).read()
    uri = re.findall(r’/song/d+’, html, re.M)
    lst = []
    for i in uri:
    link =self. base_url+i+"/download"
    lst.insert(0, link)
    lst.reverse()
    for k in lst:
    if self.rate>0:
    res = self.tieba.urlopen(k)
    else:
    res=urllib.urlopen(k).read()

    down= re.findall(r’http:.+\/music\/.+[0-9a-z]’ ,res, flags=0)
    print down
    down_url=down[self.rate].replace("\", "")

    s1 = re.search(‘title=".*’,res, re.M).group()
    s2 = re.search(‘>.*<.a’, s1, re.M).group()
    s3 = s2[1:-3]

    if self.rate==0:
    s3=unicode(s3,"utf-8","ignore").encode("gb2312","ignore")
    print s3
    urllib.urlretrieve(down_url, "music\"+s3+".mp3")
    [/python]

    python proxy

    [python]
    #!/usr/bin/python
    # Filename s5.py
    # Python Dynamic Socks5 Proxy
    # Usage: python s5.py 1080
    # Background Run: nohup python s5.py 1080 &
    # Email: ringzero@557.im

    import socket, sys, select, SocketServer, struct, time

    class ThreadingTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer): pass
    class Socks5Server(SocketServer.StreamRequestHandler):
    def handle_tcp(self, sock, remote):
    fdset = [sock, remote]
    while True:
    r, w, e = select.select(fdset, [], [])
    if sock in r:
    if remote.send(sock.recv(4096)) <= 0: break
    if remote in r:
    if sock.send(remote.recv(4096)) <= 0: break
    def handle(self):
    try:
    pass # print ‘from ‘, self.client_address nothing to do.
    sock = self.connection
    # 1. Version
    sock.recv(262)
    sock.send("x05x00");
    # 2. Request
    data = self.rfile.read(4)
    mode = ord(data[1])
    addrtype = ord(data[3])
    if addrtype == 1: # IPv4
    addr = socket.inet_ntoa(self.rfile.read(4))
    elif addrtype == 3: # Domain name
    addr = self.rfile.read(ord(sock.recv(1)[0]))
    port = struct.unpack(‘>H’, self.rfile.read(2))
    reply = "x05x00x00x01"
    try:
    if mode == 1: # 1. Tcp connect
    remote = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    remote.connect((addr, port[0]))
    pass # print ‘To’, addr, port[0] nothing do to.
    else:
    reply = "x05x07x00x01" # Command not supported
    local = remote.getsockname()
    reply += socket.inet_aton(local[0]) + struct.pack(">H", local[1])
    except socket.error:
    # Connection refused
    reply = ‘x05x05x00x01x00x00x00x00x00x00’
    sock.send(reply)
    # 3. Transfering
    if reply[1] == ‘x00’: # Success
    if mode == 1: # 1. Tcp connect
    self.handle_tcp(sock, remote)
    except socket.error:
    pass #print ‘error’ nothing to do .
    except IndexError:
    pass
    def main():
    filename = sys.argv[0];
    if len(sys.argv)<2:
    print ‘usage: ‘ + filename + ‘ port’
    sys.exit()
    socks_port = int(sys.argv[1]);
    server = ThreadingTCPServer((”, socks_port), Socks5Server)
    print ‘bind port: %d’ % socks_port + ‘ ok!’
    server.serve_forever()
    if __name__ == ‘__main__’:
    main()
    [/python]
    [python]
    # encoding=utf-8
    # Usage: python filename.py
    # Background Run: nohup python filename.py 2079 &
    # http://yaonie.org/

    import socket, thread, select, sys

    BUFLEN = 8192
    HTTPVER = ‘HTTP/1.1′

    class ConnectionHandler:
    def __init__(self, connection, address, timeout):
    self.client = connection
    self.client_buffer = ”
    self.timeout = timeout
    self.method, self.path, self.protocol = self.get_base_header()
    if self.method==’CONNECT’:
    self.method_CONNECT()
    elif self.method in (‘OPTIONS’, ‘GET’, ‘HEAD’, ‘POST’,):# ‘PUT’,’DELETE’, ‘TRACE’):
    self.method_others()
    self.client.close()
    self.target.close()

    def get_base_header(self):
    while 1:
    self.client_buffer += self.client.recv(BUFLEN)
    end = self.client_buffer.find(‘n’)
    if end!=-1:
    break
    print ‘%s’%self.client_buffer[:end]#debug
    data = (self.client_buffer[:end+1]).split()
    self.client_buffer = self.client_buffer[end+1:]
    return data

    def method_CONNECT(self):
    self._connect_target(self.path)
    self.client.send(HTTPVER+’ 200 Connection establishednProxy-agent: %snn’) %
    r"Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Kubuntu)’)"
    self.client_buffer = ”
    self._read_write()

    def method_others(self):
    self.path = self.path[7:]
    i = self.path.find(‘/’)
    host = self.path[:i]
    path = self.path[i:]
    self._connect_target(host)
    self.target.send(‘%s %s %sn’%(self.method, path, self.protocol)+self.client_buffer)
    self.client_buffer = ”
    self._read_write()

    def _connect_target(self, host):
    i = host.find(‘:’)
    if i!=-1:
    port = int(host[i+1:])
    host = host[:i]
    else:
    port = 80
    (soc_family, _, _, _, address) = socket.getaddrinfo(host, port)[0]
    self.target = socket.socket(soc_family)
    self.target.connect(address)

    def _read_write(self):
    time_out_max = self.timeout/3
    socs = [self.client, self.target]
    count = 0
    while 1:
    count += 1
    (recv, _, error) = select.select(socs, [], socs, 3)
    if error:
    break
    if recv:
    for in_ in recv:
    data = in_.recv(BUFLEN)
    if in_ is self.client:
    out = self.target
    else:
    out = self.client
    if data:
    out.send(data)
    count = 0
    if count == time_out_max:
    break

    def start_server(host, port, IPv6=False, timeout=60, handler=ConnectionHandler):
    if IPv6==True:
    soc_type=socket.AF_INET6
    else:
    soc_type=socket.AF_INET
    soc = socket.socket(soc_type)
    soc.bind((host, port))
    print "Serving on %s:%d."%(host, port)#debug
    soc.listen(0)
    while 1:
    thread.start_new_thread(handler, soc.accept()+(timeout,))

    if __name__ == ‘__main__’:
    if len(sys.argv) != 2:
    print ‘usage: python %s port’ % sys.argv[0]
    sys.exit()
    try:
    port = int(sys.argv[1])
    except:
    print ‘usage: python %s port’ % sys.argv[0]
    sys.exit()

    start_server(‘10.1.14.2’,port)
    [/python]

    Python的静态方法和类成员方法

    Python的静态方法和类成员方法都可以被类或实例访问,两者概念不容易理清,但还是有区别的:

    1. 静态方法无需传入self参数,类成员方法需传入代表本类的cls参数;
    2. 从第1条,静态方法是无法访问实例变量的,而类成员方法也同样无法访问实例变量,但可以访问类变量;
    3. 静态方法有点像函数工具库的作用,而类成员方法则更接近类似Java面向对象概念中的静态方法;

    [python]
    class MyClass:
    val1 = ‘Value 1’
    def __init__(self):
    self.val2 = ‘Value 2’

    @staticmethod
    def staticmd():
    print ‘静态方法,无法访问val1和val2’

    @classmethod
    def classmd(cls):
    print ‘类方法,类:’ + str(cls) + ‘,val1:’ + cls.val1 + ‘,无法访问val2的值’
    [/python]

     

    结论

    如果上述执行过程太复杂,记住以下两点就好了:

    静态方法:无法访问类属性、实例属性,相当于一个相对独立的方法,跟类其实没什么关系,换个角度来讲,其实就是放在一个类的作用域里的函数而已。

    类成员方法:可以访问类属性,无法访问实例属性。上述的变量val1,在类里是类变量,在实例中又是实例变量,所以容易混淆。