把这里的题目争取刷一遍
博客记录html
python的ORM框架peewee
SQLAlchemy
psycopg2
Djangojava
在1 的基础上,重点突出本身之前没注意的,作到精而不杂!!!python
Python是如何进行内存管理的c++
1).对象的引用计数机制git
Python内部使用引用计数,来保持追踪内存中的对象,全部对象都有引用计数。github
引用计数增长的状况:json
一个对象分配一个新名称
将其放入一个容器中(如列表、元组或字典)
引用计数减小的状况:api
使用del语句对对象别名显示的销毁
引用超出做用域或被从新赋值
2).垃圾回收缓存
当一个对象的引用计数归零时,它将被垃圾收集机制处理掉。安全
3).内存池机制
Python提供了对内存的垃圾收集机制,可是它将不用的内存放到内存池而不是返回给操做系统:
Pymalloc机制:为了加速Python的执行效率,Python引入了一个内存池机制,用于管理对小块内存的申请和释放。
对于Python对象,如整数,浮点数和List,都有其独立的私有内存池,对象间不共享他们的内存池。也就是说若是你分配又释放了大量的整数,用于缓存这些整数的内存就不能再分配给浮点数。
from io import StringIO def out_str(): return 'out\nstring\nhello\nworld' s = StringIO(out_str()) s.getvalue() s.seek(0,2) s.write('haha') s.getvalue() # 若是去掉 s.seek(0,2) 则意味着默认指针从 0 位置开始写入,那么haha将替换掉out\n 四个字符长度 # 因此这里 seek , tell 是隐藏的俩有用的玩意儿, s.write() 调用完指针在当前位置即最后一个字符,StringIO(str) 这种指针在 0 处 ,改用seek时候用seek
# StringIO和BytesIO # stringIO 好比说,这时候,你须要对获取到的数据进行操做,可是你并不想把数据写到本地硬盘上,这时候你就能够用stringIO from io import StringIO from io import BytesIO def outputstring(): return 'string \nfrom \noutputstring \nfunction' s = outputstring() # 将函数返回的数据在内存中读 sio = StringIO(s) # 能够用StringIO自己的方法 print(sio.getvalue()) # 也能够用file-like object的方法 s = sio.readlines() for i in s: print(i.strip()) print('*'*100) # 将函数返回的数据在内存中写 sio = StringIO() sio.write(''.join(s)) # 能够用StringIO自己的方法查看 s=sio.getvalue() print(s) print('*'*100) # 若是你用file-like object的方法查看的时候,你会发现数据为空 sio = StringIO() sio.write(s) # write 会把指针丢往最后 for i in sio.readlines(): print(i.strip()) print('*'*100) # 这时候咱们须要修改下文件的指针位置 # 咱们发现能够打印出内容了 sio = StringIO() sio.write(s) sio.seek(0,0) print(sio.tell()) for i in sio.readlines(): print(i.strip()) # 这就涉及到了两个方法seek 和 tell # tell 方法获取当前文件读取指针的位置 # seek 方法,用于移动文件读写指针到指定位置,有两个参数,第一个offset: 偏移量,须要向前或向后的字节数,正为向后,负为向前;第二个whence: 可选值,默认为0,表示文件开头,1表示相对于当前的位置,2表示文件末尾 # 用seek方法时,需注意,若是你打开的文件没有用'b'的方式打开,则offset没法使用负值哦 print('*'*100) # stringIO 只能操做str,若是要操做二进制数据,就须要用到BytesIO # 上面的sio没法用seek从当前位置向前移动,这时候,咱们用'b'的方式写入数据,就能够向前移动了 bio = BytesIO() bio.write(s.encode('utf-8')) print(bio.getvalue()) bio.seek(-36,1) print(bio.tell()) for i in bio.readlines(): print(i.strip().decode('utf-8')) print(len('string \nfrom \noutputstring \nfunction'))
from __future__ import division import sys,time j = '#' for i in range(1,61): j += '#' sys.stdout.write(str(int((i/60)*100))+'% ||'+j+'->'+"\r") sys.stdout.flush() time.sleep(0.1)
动态加载一个目录下的全部模块 目录: ---test ----a.py ----b.py ---c.py c.py导入test下面的全部模块: for path in ["test"]: for i in list(set([os.path.splitext(i)[0] for i in os.listdir("./"+path)])): if i!="__init__" and i!=".DS_Store": ##排除没必要要的文件 import_string = "import path+"."+i+" exec import_string #执行字符串中的内容
# 三方法去重 # 方法一利用 set 集合不重复特性 >>> list1 = [2,3,8,4,9,5,6] >>> list2 = [5,6,10,17,11,2] >>> set(sorted(list1)+sorted(list2)) {2, 3, 4, 5, 6, 8, 9, 10, 11, 17} # 方法二利用 dict 的 keys 不能重复的特色 >>> {}.fromkeys(sorted(list1)+sorted(list2)).keys() dict_keys([2, 3, 4, 5, 6, 8, 9, 10, 11, 17]) # 方法三,自定义函数判断若是重复则删除 >>> def distinct(lst): ... lst.sort() ... last = lst[-1] ... for i in range(len(lst)-2,-1,-1): ... if last == lst[i]: ... del lst[i] ... else: ... last = lst[i] ... return lst ... >>> distinct(list1+list2) [2, 3, 4, 5, 6, 8, 9, 10, 11, 17]
python set集合运算(交集,并集,差集,对称差集)
1》交集
x={1,2,3,4}
y={3,4,5,6}
x
set([1, 2, 3, 4])
y
set([3, 4, 5, 6])
x&y
set([3, 4])
x.intersection(y)
set([3, 4])
2》并集
x | y #集合并集
set([1, 2, 3, 4, 5, 6])
x.union(y)
set([1, 2, 3, 4, 5, 6])
3》差集
x-y # x与y的差集
set([1, 2])
x.difference(y)# x与y的差集
set([1, 2])
y-x # y与x的差集
set([5, 6])
y.difference(x)# y与x的差集
set([5, 6])
4》对称差集
x^y
set([1, 2, 5, 6])
y^x
set([1, 2, 5, 6])
x.symmetric_difference(y)
set([1, 2, 5, 6])
y.symmetric_difference(x)
set([1, 2, 5, 6])
5》集合的子集和超集
x
set([1, 2, 3, 4])
z
set([1, 2, 3])
z.issubset(x)#z是x的子集
True
x.issuperset(z)#x是z的超集
True
下面的图片形象地展现了set集合的各类运算:
集合 x <==> ① + ②
集合 x <==> ② + ③
交集 x&6 <==> ②
并集 x|y <==> ① + ② + ③
差集 x-y <==> ①
差集 y-x <==> ③
对称差集 x^y == y^x <==> ① + ③
# 本身想的一个 随机生成一个 max 位数的 生成器,也许没什么用,可是既然想了就写上吧。。。 >>> import random >>> def g_rand_range(max): ... for i in range(1,max+1,1): ... yield random.randint(1,10) ... >>> for i in g_rand_range(10): ... print(i)
#下面的代码会不会报错 list = ['a', 'b', 'c', 'd', 'e'] print(list[10:]) 答案是并不会。。 当试图访问一个超过列表索引值的成员将致使 IndexError(好比访问以上列表的 list[10])。尽管如此,试图访问一个列表的以超出列表长度数做为开始索引的切片将不会致使 IndexError,而且将仅仅返回一个空列表
# 猜猜下面执行后的结果: def extendList(val, list=[]): list.append(val) return list list1 = extendList(10) # list 为可变类型 10 list2 = extendList(123,[]) # 123 list3 = extendList('a') # 10,'a' 而这个地方 list3 再次操做了 list1 时候的那个 列表 连带 list1 都变成了 10,'a' print ("list1 = %s" % list1) print ("list2 = %s" % list2) print ("list3 = %s" % list3) list1 = [10, 'a'] list2 = [123] list3 = [10, 'a'] 为了不这样的问题, 能够 采用如下作法,参数列表里赋值为None def extendList(val, list=None): if list is None: list = [] list.append(val) return list list1 = extendList(10) # list 为可变类型 10 list2 = extendList(123,[]) # 123 list3 = extendList('a') # 10,'a' print ("list1 = %s" % list1) print ("list2 = %s" % list2) print ("list3 = %s" % list3) list1 = [10] list2 = [123] list3 = ['a']
>>>all([1<2,3>2]) True >>> any([1<2,3<2]) True >>>
# 反序一下 a = [0, 1, 2, 3, 4, 5, 7, 8, 9, 10] >>> ','.join(map(str,a[::-1]))
## 问题 @classmethod 与 @staticmethod 的区别,为啥要用这些装饰器 # 我看完举例后的浅显理解是 classmethod 解决了 c++ , java 中重载的问题, #@staticmethod 能够不用建立对象直接类调用而且可以被其子类轻易继承。。。 class Date(object): def __init__(self, day=0, month=0, year=0): self.day = day self.month = month self.year = year @classmethod def from_string(cls, date_as_string): day, month, year = map(int, date_as_string.split('-')) date1 = cls(day, month, year) return date1 @staticmethod def is_date_valid(date_as_string): day, month, year = map(int, date_as_string.split('-')) return day <= 31 and month <= 12 and year <= 3999 date2 = Date.from_string('11-09-2012') is_date = Date.is_date_valid('11-09-2012') # 改写一下 加深理解 import re class Date(object): def __init__(self,day=0,month=0,year=0): self.day = day self.month = month self.year = year @staticmethod def split_date_string(delimeter,date_as_string): day,month,year = map(int,date_as_string.split(delimeter)) return day,month,year @staticmethod def is_valid_date(date_as_string,delimeter): r_string = r'\d{1,2}%s\d{1,2}%s\d{4}' %(delimeter,delimeter) print(r_string) if re.match(r_string,date_as_string): day,month,year = Date.split_date_string(delimeter,date_as_string) else: raise ValueError('不支持的日期格式') return day<=31 and month<=12 and year<=3999 @classmethod def from_string(cls,date_as_string,delimeter): if cls.is_valid_date(date_as_string,delimeter): day,month,year = cls.split_date_string(delimeter,date_as_string) return cls(day,month,year) def __str__(self): return '/'.join(list(map(str,[self.day,self.month,self.year]))) date2 = Date.from_string('01/09/2012','/') date3 = Date.from_string('01-09-2012','-') print(date2) print(date3)
#new 相似于 java 的懒汉式 单例,判断是否有,没有就建立 # 第一种方法 new 方法 class Singleton(object): def __new__(cls,*args,**kw): if not hasattr(cls,'_instance'): cls._instance = super(Singleton,cls).__new__(cls,*args,**kw) return cls._instance s1 = Singleton() s2 = Singleton() s1 == s2 class Singleton(object): def __new__(cls, *args, **kw): if not hasattr(cls, '_instance'): orig = super(Singleton, cls) cls._instance = orig.__new__(cls, *args, **kw) return cls._instance def __init__(self,*args,**kw): pass s1 = Singleton() s2 = Singleton() print(s1==s2) print(s1 is s2) print(type(s1),'--',type(s2)) # 装饰器 ## 函数式装饰器 def singleton(cls,*args,**kw): def get_instance(): instance = {} if cls not in instance: instance[cls] = cls(*args,**kw) return instance[cls] return get_instance @singleton class Class_Demo(object): def __new__(cls,*args,**kw): pass c1 = Class_Demo() c2 = Class_Demo() c1 is c2 ## 类装饰器 # 共享属性 class Borg(object): _state = {} def __new__(cls, *args, **kw): ob = super(Borg, cls).__new__(cls, *args, **kw) ob.__dict__ = cls._state return ob class MyClass(Borg): def __new__(cls,*args,**kw): pass c1 = MyClass() c2 = MyClass() c1 is c2
Python中9种生成新对象的方法 先定义一个类: class Point: def __init__(self, x, y): self.x = x self.y = y 下面咱们使用9种方法来生成新的对象: point1 = Point(1, 2) point2 = eval("{}({}, {})".format("Point", 1, 2)) point3 = globals()["Point"](1, 2) point4 = locals()["Point"](1, 2) point5 = getattr(sys.modules[__name__], "Point")(1, 2) point6 = copy.deepcopy(point1) point7 = point1.__class__(1, 2) point8 = type('Point', (Point, ), {})(1, 2) point9 = types.new_class('Point', (Point, ), {})(1, 2
阅读下面的代码,它的输出结果是什么? class Node(object): def __init__(self,sName): self._lChildren = [] self.sName = sName def __repr__(self): return "<Node '{}'>".format(self.sName) def append(self,*args,**kwargs): self._lChildren.append(*args,**kwargs) def print_all_1(self): print self for oChild in self._lChildren: oChild.print_all_1() def print_all_2(self): def gen(o): lAll = [o,] while lAll: oNext = lAll.pop(0) lAll.extend(oNext._lChildren) yield oNext for oNode in gen(self): print oNode oRoot = Node("root") oChild1 = Node("child1") oChild2 = Node("child2") oChild3 = Node("child3") oChild4 = Node("child4") oChild5 = Node("child5") oChild6 = Node("child6") oChild7 = Node("child7") oChild8 = Node("child8") oChild9 = Node("child9") oChild10 = Node("child10") oRoot.append(oChild1) oRoot.append(oChild2) oRoot.append(oChild3) oChild1.append(oChild4) oChild1.append(oChild5) oChild2.append(oChild6) oChild4.append(oChild7) oChild3.append(oChild8) oChild3.append(oChild9) oChild6.append(oChild10) # 说明下面代码的输出结果 oRoot.print_all_1() oRoot.print_all_2() 答案 oRoot.print_all_1()会打印下面的结果: <Node 'root'> <Node 'child1'> <Node 'child4'> <Node 'child7'> <Node 'child5'> <Node 'child2'> <Node 'child6'> <Node 'child10'> <Node 'child3'> <Node 'child8'> <Node 'child9'> oRoot.print_all_1()会打印下面的结果: <Node 'root'> <Node 'child1'> <Node 'child2'> <Node 'child3'> <Node 'child4'> <Node 'child5'> <Node 'child6'> <Node 'child8'> <Node 'child9'> <Node 'child7'> <Node 'child10'> 为何提这个问题? 由于对象的精髓就在于组合(composition)与对象构造(object construction)。对象须要有组合成分构成,并且得以某种方式初始化。这里也涉及到递归和生成器(generator)的使用。 生成器是很棒的数据类型。你能够只经过构造一个很长的列表,而后打印列表的内容,就能够取得与print_all_2相似的功能。生成器还有一个好处,就是不用占据不少内存。 有一点还值得指出,就是print_all_1会以深度优先(depth-first)的方式遍历树(tree),而print_all_2则是宽度优先(width-first)。有时候,一种遍历方式比另外一种更合适。但这要看你的应用的具体状况。
# 零宽断言,举例, (?=),(?<=),(?!),(?<!) import re s = 'abc123ebo456' # re.findall(r'[a-z]+(?=\d+)',s) # ['abc', 'ebo'] 先行零宽断言 后面是 数字,findall 取出 字母部分 # re.findall(r'(?<=[a-z])\d+',s) # ['123', '456'] 后发零宽断言 前面是 字母, findall 取出 \d+ 部分 # re.findall(r'[a-z]+(?!c)',s) # ['abc', 'ebo']负向零宽先行断言 后面不是 c 这个 c 随便取的,能够为 a只要不是数字 # re.findall(r'(?<!\d)\d+',s) # 负向零宽后发断言 ['123', '456']
好比要匹配2个一样的字符 '(\w)\1',\1是对(\w)的引用。 这样有个弊端,就是分组不少,并且有嵌套的时候,很难分清\n引用的是哪一个分组。 用命名分组就直观多了。 '(?P<word>\w)(?P=word)' import re s = 'abc1334oioj33435lljlj45' re.findall(r'(?P<repeatdigit>\d+).*?(?P=repeatdigit)',s) # ['334', '5']
import re s = 'abc' re.findall(r'(?:a)(.*)',s) # ['bc']
lock = threading.Lock() with lock: print('Citical part...do something...')
# -*- coding: utf-8 -*- def func_cache(func): cache = {} def inner_deco(*args): if args in cache: print('func {} is already cached with arguments {}'.format( func.__name__, args)) return cache[args] else: print('func {} is not cached with arguments {}'.format( func.__name__, args)) res = func(*args) cache[args] = res return res return inner_deco @func_cache def add_two_number(a, b): return a + b if __name__ == "__main__": print('1. add_two_number(1, 2)') add_two_number(1, 2) print('2. add_two_number(2, 3)') add_two_number(2, 3) print('3. add_two_number(1, 2)') add_two_number(1, 2)
import re from collections import Counter words_list = [] with open('/home/ml/work/notebook/Frank Li/words.txt','r') as fr: for line in fr: words_array = line.strip().split(" ") for w in words_array: words_list.append(w) words_list.remove('') Counter(words_list) {w:words_list.count(w) for w in words_list} >>> from collections import Counter >>> list_1 = [1,2,3,1,2,4,5,6,7,4,8,9] >>> [key for key,cnt in Counter(list_1).items() if cnt>1] [1, 2, 4]
import re from collections import Counter words_list = [] with open('/home/ml/work/notebook/Frank Li/words.txt','r') as fr: for line in fr: words_array = line.strip().split(" ") words_list.extend(words_array) # for w in words_array: # words_list.append(w) def flattern(words_lst): for lst in words_lst: if isinstance(lst,list): tmp = flattern(lst) for i in tmp: yield i else: yield lst words_list = list(flattern(words_list)) words_list.remove('') Counter(words_list) {w:words_list.count(w) for w in words_list}
BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception +-- StopIteration +-- StandardError | +-- BufferError | +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning
# -*- coding: utf-8 -*- """ requests.api ~~~~~~~~~~~~ This module implements the Requests API. :copyright: (c) 2012 by Kenneth Reitz. :license: Apache2, see LICENSE for more details. """ from . import sessions def request(method, url, **kwargs): """Constructs and sends a :class:`Request <Request>`. :param method: method for the new :class:`Request` object. :param url: URL for the new :class:`Request` object. :param params: (optional) Dictionary, list of tuples or bytes to send in the body of the :class:`Request`. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`. :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`. :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`. :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload. ``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')`` or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers to add for the file. :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth. :param timeout: (optional) How many seconds to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple :param allow_redirects: (optional) Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to ``True``. :type allow_redirects: bool :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy. :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use. Defaults to ``True``. :param stream: (optional) if ``False``, the response content will be immediately downloaded. :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair. :return: :class:`Response <Response>` object :rtype: requests.Response Usage:: >>> import requests >>> req = requests.request('GET', 'http://httpbin.org/get') <Response [200]> """ # By using the 'with' statement we are sure the session is closed, thus we # avoid leaving sockets open which can trigger a ResourceWarning in some # cases, and look like a memory leak in others. with sessions.Session() as session: return session.request(method=method, url=url, **kwargs) def get(url, params=None, **kwargs): r"""Sends a GET request. :param url: URL for the new :class:`Request` object. :param params: (optional) Dictionary, list of tuples or bytes to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ kwargs.setdefault('allow_redirects', True) return request('get', url, params=params, **kwargs) def options(url, **kwargs): r"""Sends an OPTIONS request. :param url: URL for the new :class:`Request` object. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ kwargs.setdefault('allow_redirects', True) return request('options', url, **kwargs) def head(url, **kwargs): r"""Sends a HEAD request. :param url: URL for the new :class:`Request` object. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ kwargs.setdefault('allow_redirects', False) return request('head', url, **kwargs) def post(url, data=None, json=None, **kwargs): r"""Sends a POST request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json data to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('post', url, data=data, json=json, **kwargs) def put(url, data=None, **kwargs): r"""Sends a PUT request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json data to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('put', url, data=data, **kwargs) def patch(url, data=None, **kwargs): r"""Sends a PATCH request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json data to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('patch', url, data=data, **kwargs) def delete(url, **kwargs): r"""Sends a DELETE request. :param url: URL for the new :class:`Request` object. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('delete', url, **kwargs)