资源

Outline：

Jiaqi-knight’s repositories
.ipynb_checkpoints
00-Python Object and Data Structure Basics
01-Python Comparison Operators
02-Python Statements
03-Methods and Functions
04-Milestone Project - 1
05-Object Oriented Programming
06-Modules and Packages
07-Errors and Exception Handling
08-Milestone Project - 2
09-Built-in Functions
10-Python Decorators
11-Python Generators
12-Advanced Python Modules
12-Final Capstone Python Project
13-Advanced Python Modules
13-Web-Scraping
14-Working-with-Images
15-Advanced OOP
15-PDFs-and-Spreadsheets
16-Milestone Project - 3
17-Advanced Python Objects and Data Structures
17-Parallel Processing
18-Bonus Material - Introduction to GUIs
FAQ.ipynb
Jupyter (iPython) Notebooks Guide.ipynb Notebooks Guide.ipynb)
Untitled.ipynb
README.md

环境配置

  • 如何管理不同版本的python pyenv (m1目前能下载，但python版本安装无法完成，不建议)

  • Anaconda 路径设置 

    1
    export PATH="/opt/anaconda3/bin:$PATH"   #或者直接加入到~/.zshrc中
    1
    export PATH="/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin:$PATH" #python3.8-dev

  • 使用项目尽量创建虚拟环境，以免污染系统python环境

  • 参考M1 python环境配置-在conda中配置python3.9环境，然后通过conda install -c conda-forge scikit-learn安装sklearn

Python对象与数据结构

数据类型

Integers int 2 3 4
Floating Point float 2.3 4.0
Strings str “hello”
Lists list [10,”hello”,200.2] sequence
Dictionaries dict {“mykey”: “value”, “name”:”Franklie”}
Tuples tup (10,”hello”,200.3) immutable sequence(not work: name[0] =’P’)
Sets set {“a”,”b”}
Booleans bool TRUE or FLASE

变量赋值

变量名规则

  • 名称不能以数字开头

  • 名称不能包含空格，请使用_ intead

  • 名称不能包含以下任何符号：

    1
    :'",<>/?|\!@#%^&*~-+

  • 最佳做法（PEP8）被认为是小写且带有下划线的名称

  • 避免使用Python内置关键字，例如list和str

  • 避免使用单个字符l（小写字母el），O（大写字母oh）和I（大写字母眼睛），因为它们可能与1和混淆0

用type()来确定变量类型

String

用字符串建立索引和切片

1
2
3
4
5
6
7
8
9
10
mystring = 'abcdefh'
mystring[:3]    # 前 0，1，2字符，不包含第3个字符
mystring[2：]   # 不包含 0，1，包含第2个字符 
mystring[3:6]   # 包含第3个字符，不包含第6个字符 （从0开始数）
mystring[::2]   # 0,2,4,...
mystring[2:7:2] # 2,4,6 (2和7之间，2个间隔)
mystring[::-1]  # reverse

'Hello'[0] # returns 'H'

String Method

String Properties and Methods

1
2
3
4
5
6
7
# string is immutable(not work: name[0] ='P')
'P' + 'J' # = 'PJ'
x.upper() # 大写
x.lower() # 小写
x.split() # split by 空格
x.split('i') # split by “i”

Formatting with the .format() method

1
2
3
4
5
6
7
print('a {} {}'.format('string','Insert'))   # Out：a string Insert
print('a {1} {0}'.format('string','Insert')) # Out：a Insert string 
print('a {b} {f}'.format(f='string',b='Insert')) # Out：a Insert string 
# 简化
name = 'Same'
age  = 3
print(f'{name} is {age} years old.') # Out: Same is 3 years old.

Float formatting follows “{value:width.precision f}” 指定输出的数值精度

1
2
3
4
5
result = 100/777 
# Out: 0.12870012870012876
print("The result was {r:1.3f}".format(r=result)) 
# Out: The result was 0.129 #保留小数点后三位有效数字

更多用途请参考:

Method Description
capitalize() Converts the first character to upper case
casefold() Converts string into lower case
center() Returns a centered string
count() Returns the number of times a specified value occurs in a string
encode() Returns an encoded version of the string
endswith() Returns true if the string ends with the specified value
expandtabs() Sets the tab size of the string
find() Searches the string for a specified value and returns the position of where it was found
format() Formats specified values in a string
format_map() Formats specified values in a string
index() Searches the string for a specified value and returns the position of where it was found
isalnum() Returns True if all characters in the string are alphanumeric
isalpha() Returns True if all characters in the string are in the alphabet
isdecimal() Returns True if all characters in the string are decimals
isdigit() Returns True if all characters in the string are digits
isidentifier() Returns True if the string is an identifier
islower() Returns True if all characters in the string are lower case
isnumeric() Returns True if all characters in the string are numeric
isprintable() Returns True if all characters in the string are printable
isspace() Returns True if all characters in the string are whitespaces
istitle() Returns True if the string follows the rules of a title
isupper() Returns True if all characters in the string are upper case
join() Joins the elements of an iterable to the end of the string
ljust() Returns a left justified version of the string
lower() Converts a string into lower case
lstrip() Returns a left trim version of the string
maketrans() Returns a translation table to be used in translations
partition() Returns a tuple where the string is parted into three parts
replace() Returns a string where a specified value is replaced with a specified value
rfind() Searches the string for a specified value and returns the last position of where it was found
rindex() Searches the string for a specified value and returns the last position of where it was found
rjust() Returns a right justified version of the string
rpartition() Returns a tuple where the string is parted into three parts
rsplit() Splits the string at the specified separator, and returns a list
rstrip() Returns a right trim version of the string
split() Splits the string at the specified separator, and returns a list
splitlines() Splits the string at line breaks and returns a list
startswith() Returns true if the string starts with the specified value
strip() Returns a trimmed version of the string
swapcase() Swaps cases, lower case becomes upper case and vice versa
title() Converts the first character of each word to upper case
translate() Returns a translated string
upper() Converts a string into upper case
zfill() Fills the string with a specified number of 0 values at the beginning

List

  • Lists are ordered sequences that can hold a variety of object types.
  • They use [] brackets and commas to separate objects in the list。
  • List support indexing and slicing. Lists can be nested and also have a variety of useful methods that can be called off of them.
1
2
3
my_list = [1,2,3]
my_list = ['STRING', 100, 23.4]
len(my_list) #length of list

Slicing is just like string except:

1
2
3
4
my_list = ['one','two','three']
my_list[0] = 'list can be changed'
my_list
#Out： ['list can be changed','two','three']

List Methods

tab4method

1
my_list.append('four') # Out:['one','two','three','four']
Method Description
append() Adds an element at the end of the list
clear() Removes all the elements from the list
copy() Returns a copy of the list
count() Returns the number of elements with the specified value
extend() Add the elements of a list (or any iterable), to the end of the current list
index() Returns the index of the first element with the specified value
insert() Adds an element at the specified position
pop() Removes the element at the specified position
remove() Removes the first item with the specified value
reverse() Reverses the order of the list
sort() Sorts the list

Dictionary

  • Dictionaries are unordered mappings for storing objects. Previously we saw how lists store objects in an ordered sequence, dictionaries use a key-value pairing instead.
  • This key-value pair allows users to quickly grab objects without needing to know an index location.
  • Dictionaries use curly braces and colons to signify the keys and their associated values.
  • So when to choose a list and when to choose a dictionary？
1
2
my_dict={'key1':'value1','key2':'value2'}
my_dict['key1'] #Out：'value1'
1
2
3
4
d = {'k1':123, 'k2':[0,2,3],'k3':{'insideKey':100}} # a dict inside a dict
d['k3']['insideKey'] # Out: 100
d['k2'][2]           # Out: 2

Dictionary methods

Method Description
clear() Removes all the elements from the dictionary
copy() Returns a copy of the dictionary
fromkeys() Returns a dictionary with the specified keys and value
get() Returns the value of the specified key
items() Returns a list containing a tuple for each key value pair
keys() Returns a list containing the dictionary’s keys
pop() Removes the element with the specified key
popitem() Removes the last inserted key-value pair
setdefault() Returns the value of the specified key. If the key does not exist: insert the key, with the specified value
update() Updates the dictionary with the specified key-value pairs
values() Returns a list of all the values in the dictionary

Tuples

  • Tuples are very similar to lists. However they have one key different - immutability.
  • Once an element is inside a tuple, it can not be reassigned.
  • Tuples use parenthesis括号： (1,2,3)
Method Description
count() Returns the number of times a specified value occurs in a tuple
index() Searches the tuple for a specified value and returns the position of where it was found

Sets

  • Sets are unordered collections of only accepted unique elements.
  • only accept 
1
2
3
4
mylist = [1,2,1,1,1,2,2,2]
ser(mylist) #Out:{1,2} only unique elements output
set('mmmssaq') # Out:{'a', 'm', 'q', 's'}

Method Description
add() Adds an element to the set
clear() Removes all the elements from the set
copy() Returns a copy of the set
difference() Returns a set containing the difference between two or more sets
difference_update() Removes the items in this set that are also included in another, specified set
discard() Remove the specified item
intersection() Returns a set, that is the intersection of two other sets
intersection_update() Removes the items in this set that are not present in other, specified set(s)
isdisjoint() Returns whether two sets have a intersection or not
issubset() Returns whether another set contains this set or not
issuperset() Returns whether this set contains another set or not
pop() Removes an element from the set
remove() Removes the specified element
symmetric_difference() Returns a set with the symmetric differences of two sets
symmetric_difference_update() inserts the symmetric differences from this set and another
union() Return a set containing the union of sets
update() Update the set with the union of this set and others

Booleans

  • Booleans are operators that allow you to convey True or False statements.

File

Method Description
close() Closes the file
detach() Returns the separated raw stream from the buffer
fileno() Returns a number that represents the stream, from the operating system’s perspective
flush() Flushes the internal buffer
isatty() Returns whether the file stream is interactive or not
read() Returns the file content
readable() Returns whether the file stream can be read or not
readline() Returns one line from the file
readlines() Returns a list of lines from the file
seek() Change the file position
seekable() Returns whether the file allows us to change the file position
tell() Returns the current file position
truncate() Resizes the file to a specified size
writable() Returns whether the file can be written to or not
write() Writes the specified string to the file
writelines() Writes a list of strings to the file
writefile %%writefile myfile.txt 
1
2
3
4
5
x = open('test.txt','w')
x.write('Hello World')
x.close()
with open('test.txt') as my_new_file
	contents = my_new_file.read()
  • mode = ‘r’ is read only
  • mode = ‘w’ is write only (will overwrite files or create new!)
  • mode = ‘a’ is append only (will add on to files)
  • mode = ‘r+’ is reading and writing
  • mode = ‘w+’ is writing and reading (Overwrites existing files or creates a new file!)

Python Statements

If else

1
2
3
4
5
6
7
8
9
10
#Code in this cell
for i in range(1,101):
    if (i%3==0) and (i%5!=0):
        print('Fizz')
    elif (i%3!=0) and (i%5==0):
        print('Buzz')
    elif (i%3==0) and (i%5==0):
        print('FizzBuzz')
    else:
        print(i)

for loop

1
2
3
4
5
6
7
8
my_iterable = [1,2,3]
for item_name in my_iterable: 
        			print(item_name)
for num in range(10)
	print(nunm) #Out:1,2...,9
for num in range(0,11,2)
	print(nunm) #Out:0,2,4,6,8,10

1
2
3
4
5
6
7
8
9
10
11
index_count = 0
word = 'abcde'
for letter in word:
    print('An index {} the letter is {}'.format(index_count, letter))
    index_count += 1
# Out:
An index 0 the letter is a
An index 1 the letter is b
An index 2 the letter is c
An index 3 the letter is d
An index 4 the letter is e
1
2
3
4
5
6
7
8
9
10
word = 'abcde'
for item in enumerate(word):
    print(item)
# Out:
(0, 'a')
(1, 'b')
(2, 'c')
(3, 'd')
(4, 'e')

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
word = 'abcde'
for item,letter in enumerate(word):
    print(item)
    print(letter)
# Out:
0
a
1
b
2
c
3
d
4
e

1
2
3
4
5
6
7
8
9
mylist1 = [1,2,3]
mylist2 = ['a','b','c']
mylist3 = [100,200,300]
for item in zip(mylist1,mylist2,mylist3):
	print(item)
#Out: list[tuple,tuple,tuple]
(1, 'a', 100)
(2, 'b', 200)
(3, 'c', 300)

while else

1
2
while some_boolean_condition:         				
         #do something

Useful Operator

1
2
3
4
5
6
7
8
9
10
11
'x' in ['x','y','z'] # Out: true
'x' in [1,2,3]       # Out: false
'x' not in ['x','y','z'] #Out:False
min([10,20,30]) # Out：10
max([10,20,30]) # Out：30

from random import randint
randint(0,100) # return a int

 
input('Enter Something into this box: ') # input

List Comprehensions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mystring = 'hello'
mylist   = []
for letter in mystring:
	mylist.append(letter)
	
mylist
# return: ['h','e','l','l','o']

# !More effcient way!

mylist   = [leeter for letter in mystring]

mylist
# return: ['h','e','l','l','o']
1
2
3
mylist = [num**2 for num in range(0,11)]
mylist
# return: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
1
2
3
mylist = [num**2 for num in range(0,11) if num%2==0]
mylist
# return: [0, 4, 16, 36, 64, 100]
1
2
3
4
celcius = [0,10,20,34.5]
fahrenheit = [( (9/5)*temp + 32) for temp in celcius]
fahrenheit
# return: [32.0, 50.0, 68.0, 94.1]
1
2
3
mylist = [num if num%2==0 else 'ODD' for num in range(0,11) ]
mylist
# return: [0, 'ODD', 2, 'ODD', 4, 'ODD', 6, 'ODD', 8, 'ODD', 10]

Methods and Functions

1
help(mylist.pop()) # help of method with ducuments

tab4method

  • Creating a function requires a very specific syntax, including the def keyword, correct indentation, and proper structure.
  • Typically we use the return keyword to send back the result of the function, instead of just printing it out.

Structure of function

1
2
3
def add_num(num1, num2):
    return num1+num2
add_num(10,20)
1
2
3
def say_hello(name='Default'):
	print(f'Hello {name}')
say_hello('Jose')
1
2
3
def print_result(a,b):
	print(a+b) # diff with return

1
2
3
4
5
6
7
8
9
10
#偶数大写，奇数小写
def myfunc(st):
    s=''
    for index in range(len(st)):
        if index % 2 ==0:
            s = s + st[index].upper()
        else:
            s = s + st[index].lower()
    return s
    
1
2
def master_yoda(text):
    return ' '.join(text.split()[::-1])

args and *kwargs

1
2
3
4
5
def myfunc(*args):
     return sum(args） # return back a tuple

myfunc(4,6,10) # return 20

1
2
3
4
5
def myfunc(*args):
     for item in args: # return back a tuple
     	print(item)

myfunc(4,6,10) # return 4 6 10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def myfunc(**kwargs): # dictionary(类似c++指针的指针，包含map映射的关系)
    print(kwargs)
    if 'fruit' in kwargs:
        print('My fruit of choice is {}'.format(kwargs['fruit']))
    else:
        print('I did not find any fruit here')
myfunc(fruit='apple',veggie='lettuce')

# return: 
def myfunc(**kwargs): # dictionary(类似c++指针的指针，包含map映射的关系)

    print(kwargs)

    if 'fruit' in kwargs:

        print('My fruit of choice is {}'.format(kwargs['fruit']))

    else:

        print('I did not find any fruit here')

myfunc(fruit='apple',veggie='lettuce')

#return:
#{'fruit': 'apple', 'veggie': 'lettuce'}
#My fruit of choice is apple

1
2
3
4
5
6
7
8
9
10
11
def myfunc(*args,**kwargs):
    print(args)
    print(kwargs)
    print('I would like {} {}'.format(args[0],kwargs['food']))
myfunc(10,20,30,fruit='orange',food='eggs',animal='dog')

# return:
#(10, 20, 30)
#{'fruit': 'orange', 'food': 'eggs', 'animal': 'dog'}
#I would like 10 eggs

1
2
3
4
def myfunc(*args):
	return [num for num in args if num%2==0]
myfunc(5,6,7,8)
# Out:[6, 8]

一些有趣的函数技巧

PAPER DOLL: Given a string, return a string where for every character in the original there are three characters
1
2
paper_doll('Hello') --> 'HHHeeellllllooo'
paper_doll('Mississippi') --> 'MMMiiissssssiiippppppiii'
1
2
3
4
5
def paper_doll(text):
    result = ''
    for char in text:
        result += char * 3 #char + char + char
    return result
21点
1
2
3
4
5
def paper_doll(text):
    result = ''
    for char in text:
        result += char * 3
    return result

Lambda Expressions, Map, and Filter

tip：“shift+tab”查看ducumentation

1
2
3
4
def square(num):
    return num**2
my_nums = [1,2,3,4,5]
list(map(square,my_nums)) #快速映射计算，类似并行
1
2
3
4
5
def check_even(num):
    return num % 2 == 0 
nums = [0,1,2,3,4,5,6,7,8,9,10]
filter(check_even,nums)
list(filter(check_even,nums))
1
2
3
square = lambda num: num **2 #类似定义：def square(num): return num**2
list(map(lambda n: n ** 2 ,nums))      #整个iseven function 用Lambda关键字代替
list(filter(lambda n: n % 2 == 0,nums)) #整个iseven function 用Lambda关键字代替

面向对象的编程

  • Object Oriented Programming (OOP) allows programmers to create their own objects that have methods and attributes.
  • Recall that after defining a string, list, dictionary, or other objects, you were able to call methods off of them with the .method_name() syntax.
1
2
3
4
5
6
7
8
9
10
11
class NameOfClass(): #name of class
  ## CLASS OBJECT ATTRIBUTE
  ## SAME FOR INSTANCE OF A CLASS
  species = 'mammal'
   
	def __int__(self, param1, param2): #self指针 is a object
		self.param1 = param1
		self.param2 = param2
	def some_method(self):
		#perform some action
		print(self.param1)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
class Dog():
    
    species = 'mammal'

    def __init__(self,breed,name):
      	# Attributes
        # We take in the argument
        # Assign it using self.attribute_name
        self.breed = breed
        self.name  = name
        
    # OPERATORS/ACTIONS --> Methods
    def bark(self, number):
        print("WOOF! My name is {} and the number is {}".format(self.name, number))

my_dog = Dog("Lab","Frankie")
my_dog.bark(10)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class Circle():
	
	# CLASS OBJECT ATTRIBUTE
	pi = 3.14
	
	def __init__(self, radius=1): # default radius value 
		self.radius = radius
		
	# Method
	def get_circumfrece(self):
		return self.radius * self.pi *2

# Run
my_circle = Circle()
my_circle.get_circumfrece()

继承

  • 复用class
1
2
3
4
5
6
7
8
9
class Animal():
  	def __init__(self):
    	print("Animal Created")
  
  	def who_am_i(self):
    	print("I am an animal")
    
  	def eat(self):
    	print("I am eating")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class Dog(Animal): #继承一些Animal的特征
  
  	def __init__(self):
    	Animal.__init__(self)
    	print("Dog Created")
    
  	def who_am_i(self):
    	print("I am a dog")
    def bark(self):
      print("WOOF!")
   
# RUN
myDog = Dog()
myDog.who_am_i()
myDog.bark()
# Out:
Animal Created
Dog Created
I am a dog
WOOF!

多态 Polymorphism

  • Share the same methods but different classes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class Animal:
    def __init__(self, name):    # Constructor of the class
        self.name = name

    def speak(self):              # Abstract method, defined by convention only
        raise NotImplementedError("Subclass must implement abstract method")


class Dog(Animal):
    
    def speak(self):
        return self.name+' says Woof!'
    
class Cat(Animal):

    def speak(self):
        return self.name+' says Meow!'
    
fido = Dog('Fido')
isis = Cat('Isis')

print(fido.speak())
print(isis.speak())

特殊函数 in Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class Book:
    def __init__(self, title, author, pages):
        print("A book is created")
        self.title = title
        self.author = author
        self.pages = pages

    def __str__(self):
        return "Title: %s, author: %s, pages: %s" %(self.title, self.author, self.pages)

    def __len__(self):
        return self.pages

    def __del__(self):
        print("A book is destroyed")
1
2
3
4
5
6
book = Book("Python Rocks!", "Jose Portilla", 159)

#Special Methods
print(book)
print(len(book))
del book

Modules and Packages

  • pip 指令to download packages at command line directly from the PyPi repositoty.
  • Google search for “python package for XXX”
  • Prepare your Python package for publication
  • Think about versioning
  • Upload your package to PyPI

Build your own modules and packages

1
2
3
# mymodule.py
def my_func():
	print("Hey I am in mymodule.py")
1
2
3
4
#myprogram.py -- import from module
from mymodule import my_func

my_func()
1
2
3
# bash
python3 myprogram.py
#Out: Hey I am in mymodule.py

Then modules together to form a package:(too much of my module file so organized to a folder named “MyMainPackage”)

1
2
3
4
5
mkdir MyMainPackage
cd MyMainPackage
mkdir subPackage
vim subPackage/__init__.py
vim __init__.py
1
2
3
# mysubscript.py
def sub_report():
	print("Hey I am a subscript insider subPackage")
1
2
3
4
#some_main_script.py
# use the code from mymoodule.py
def report_main():
  print("Hey I am a some main script in main package")
1
2
3
4
5
6
#myprogram.py -- import from package
from MyMainPackage import some_main_script
from MyMainPackage.subPackage import mysubscript

some_main_script.report_main()
mysubscript.sub_report()
1
2
3
wjq@wjqdeMacBook-Pro Desktop % python3 myprogram.py
Hey I am a some main script in main package
Hey I am a subscript insider subPackage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
├── MyMainPackage
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-39.pyc
│   │   └── some_main_script.cpython-39.pyc
│   ├── some_main_script.py
│   └── subPackage
│       ├── __init__.py
│       ├── __pycache__
│       │   ├── __init__.cpython-39.pyc
│       │   └── mysubscript.cpython-39.pyc
│       └── mysubscript.py
├── mymodule.py
└── myprogram.py

How to publish it on Pypi website参考1 参考2参考3

接下来，让我们来尝试构在Pypi上构建自己的pacakge Step by Step.

首先查看https://pypi.org/search，命名自己的程序包，不能重名, 这里暂且命名为jiaqi-test

1
2
3
4
5
mkdir jiaqi-test && cd jiaqi-test
python3 -m venv . # set env
source bin/activate # 进入配置环境命令
(jiaqi-test) wjq@wjqdeMacBook-Pro jiaqi-test % 
mkdir square
1
2
3
4
5
6
7
8
9
# mkdir square/__main__.py
def main():
    number = int(input('Enter the number (only positive integers allowed\n'))
    print(f'{number} squares is {number**2}')


if __name__ == '__main__': 
#直接运行该文件，否则为库导入
    main()
1
2
3
4
5
# mkdir square/__init__.py

# Vesion of module

__version__ = "0.0.1"
1
2
# Test
python square
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# mkdir setup.py


import pathlib
from setuptools import setup

# The directory containing this file
HERE = pathlib.Path(__file__).parent

# The text of the README file
README = (HERE / "README.md").read_text()

# This call to setup() does all the work
setup(
    name="jiaqi-test",
    version="1.0.0",
    description="It squares the number",
    long_description=README,
    long_description_content_type="text/markdown",
    url="https://github.com/Jiaqi-knight/jiaqi-test",
    author="Jiaqi-knight",
    author_email="jiaqiwang969@gmail.com",
    license="MIT",
    classifiers=[
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python :: 3.9",

    ],
    packages=["square"],
    include_package_data=True,
    install_requires=[],
    entry_points={
        "console_scripts": [
            "square=square.__main__:main",
        ]
    }, 
)
1
2
pip install --upgrade setuptools wheel
pip install twine
1
python setup.py sdist bdist_wheel
1
2
3
twine check dist/*
Checking dist/jiaqi_test-1.0.0-py3-none-any.whl: PASSED
Checking dist/jiaqi-test-1.0.0.tar.gz: PASSED
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# 查看 dist
tar tzf dist/jiaqi-test-1.0.0.tar.gz
jiaqi-test-1.0.0/
jiaqi-test-1.0.0/MANIFEST.in
jiaqi-test-1.0.0/PKG-INFO
jiaqi-test-1.0.0/README.md
jiaqi-test-1.0.0/jiaqi_test.egg-info/
jiaqi-test-1.0.0/jiaqi_test.egg-info/PKG-INFO
jiaqi-test-1.0.0/jiaqi_test.egg-info/SOURCES.txt
jiaqi-test-1.0.0/jiaqi_test.egg-info/dependency_links.txt
jiaqi-test-1.0.0/jiaqi_test.egg-info/entry_points.txt
jiaqi-test-1.0.0/jiaqi_test.egg-info/top_level.txt
jiaqi-test-1.0.0/setup.cfg
jiaqi-test-1.0.0/setup.py
jiaqi-test-1.0.0/square/
jiaqi-test-1.0.0/square/__init__.py
jiaqi-test-1.0.0/square/__main__.py

1
2
3
4
# 上传测试 https://packaging.python.org/guides/using-testpypi/
twine upload --repository testpypi dist/*
# 然后输入注册的用户名和密码

完成！你可以登陆View at:

https://test.pypi.org/project/jiaqi-test/1.0.0/查看。

并且可以根据命令进行下载

1
pip install -i https://test.pypi.org/simple/ jiaqi-test==1.0.0

然后，就可以运行square命令。

Handling versions of your published module & other tips

首先，会尝试修改代码，在本地运行整个square库。

1
2
# version control
pip install bump2version
1
2
# 改变代码中的version number
bump2version --allow-dirty --current-version 1.0.0 --new-version 1.0.1 patch setup.py square/__init__.py 
1
2
3
#重新运行 step-8， step-11
python setup.py sdist bdist_wheel
twine upload --repository testpypi --skip-existing dist/*

完成新版更新

newVersion

重新安装新版本

1
2
3
4
5
6
7
8
9
10
11
12
#首先卸载原先的版本
pip uninstall jiaqi_test        

Found existing installation: jiaqi-test 1.0.0
Uninstalling jiaqi-test-1.0.0:
  Would remove:
    /Users/wjq/Documents/GitHub/Test/Test03/jiaqi-test/bin/square
    /Users/wjq/Documents/GitHub/Test/Test03/jiaqi-test/lib/python3.9/site-packages/jiaqi_test-1.0.0.dist-info/*
    /Users/wjq/Documents/GitHub/Test/Test03/jiaqi-test/lib/python3.9/site-packages/square/*
Proceed (y/n)? y
  Successfully uninstalled jiaqi-test-1.0.0

1
pip install -i https://test.pypi.org/simple/ jiaqi-test==1.0.2

错误处理

1
2
3
4
5
6
7
while Ture：
try： 
except:  # if not sucess
  continue #重新回到loop再次运行
else:
  break
finally: # 无论正确与否都会经过这一步

Pylint:代码风格的检查工具

例如写了一个simple.py的测试代码

1
2
3
4
5
# simple.py
a = 1
b = 1
print(a)
print(B)
1
2
3
4
5
6
7
8
9
10
11
12
# Out:
(Test04) wjq@wjqdeMacBook-Pro test04_code % pylint simple.py 

************* Module simple
simple.py:4:0: C0304: Final newline missing (missing-final-newline)
simple.py:1:0: C0114: Missing module docstring (missing-module-docstring)
simple.py:1:0: C0103: Constant name "a" doesn't conform to UPPER_CASE naming style (invalid-name)
simple.py:2:0: C0103: Constant name "b" doesn't conform to UPPER_CASE naming style (invalid-name)
simple.py:4:6: E0602: Undefined variable 'B' (undefined-variable)

-------------------------------------
Your code has been rated at -12.50/10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
''' 
A Very Simple Script
'''

def my_func():
    '''
    A simple Function
    '''

    first  = 1
    second = 2
    print( first )
    print( second )


my_func()
1
2
3
4
5
6
7
8
# 改进之后
(Test04) wjq@wjqdeMacBook-Pro test04_code % pylint simple.py
************* Module simple
simple.py:1:3: C0303: Trailing whitespace (trailing-whitespace)
simple.py:17:0: C0304: Final newline missing (missing-final-newline)

------------------------------------------------------------------
Your code has been rated at 6.67/10 (previous run: 6.67/10, +0.00)

单元测试

单元测试是用来对一个模块、一个函数或者一个类来进行正确性检验的测试工作。

比如对函数abs()，我们可以编写出以下几个测试用例：

  1. 输入正数，比如1、1.2、0.99，期待返回值与输入相同；
  2. 输入负数，比如-1、-1.2、-0.99，期待返回值与输入相反；
  3. 输入0，期待返回0；
  4. 输入非数值类型，比如None、[]、{}，期待抛出TypeError。

把上面的测试用例放到一个测试模块里，就是一个完整的单元测试。

如果单元测试通过，说明我们测试的这个函数能够正常工作。如果单元测试不通过，要么函数有bug，要么测试条件输入不正确，总之，需要修复使单元测试能够通过。

单元测试通过后有什么意义呢？如果我们对abs()函数代码做了修改，只需要再跑一遍单元测试，如果通过，说明我们的修改不会对abs()函数原有的行为造成影响，如果测试不通过，说明我们的修改与原有行为不一致，要么修改代码，要么修改测试。

这种以测试为驱动的开发模式最大的好处就是确保一个程序模块的行为符合我们设计的测试用例。在将来修改的时候，可以极大程度地保证该模块行为仍然是正确的。

我们来编写一个Dict类，这个类的行为和dict一致，但是可以通过属性来访问，用起来就像下面这样：

1
2
3
4
5
>>> d = Dict(a=1, b=2)
>>> d['a']
1
>>> d.a
1

mydict.py代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
class Dict(dict):

    def __init__(self, **kw):
        super().__init__(**kw)

    def __getattr__(self, key):
        try:
            return self[key]
        except KeyError:
            raise AttributeError(r"'Dict' object has no attribute '%s'" % key)

    def __setattr__(self, key, value):
        self[key] = value

为了编写单元测试，我们需要引入Python自带的unittest模块，编写mydict_test.py如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import unittest

from mydict import Dict

class TestDict(unittest.TestCase):

    def test_init(self):
        d = Dict(a=1, b='test')
        self.assertEqual(d.a, 1)
        self.assertEqual(d.b, 'test')
        self.assertTrue(isinstance(d, dict))

    def test_key(self):
        d = Dict()
        d['key'] = 'value'
        self.assertEqual(d.key, 'value')

    def test_attr(self):
        d = Dict()
        d.key = 'value'
        self.assertTrue('key' in d)
        self.assertEqual(d['key'], 'value')

    def test_keyerror(self):
        d = Dict()
        with self.assertRaises(KeyError):
            value = d['empty']

    def test_attrerror(self):
        d = Dict()
        with self.assertRaises(AttributeError):
            value = d.empty

编写单元测试时，我们需要编写一个测试类，从unittest.TestCase继承。

以test开头的方法就是测试方法，不以test开头的方法不被认为是测试方法，测试的时候不会被执行。

对每一类测试都需要编写一个test_xxx()方法。由于unittest.TestCase提供了很多内置的条件判断，我们只需要调用这些方法就可以断言输出是否是我们所期望的。最常用的断言就是assertEqual()：

1
self.assertEqual(abs(-1), 1) # 断言函数返回的结果与1相等

另一种重要的断言就是期待抛出指定类型的Error，比如通过d['empty']访问不存在的key时，断言会抛出KeyError：

1
2
with self.assertRaises(KeyError):
    value = d['empty']

而通过d.empty访问不存在的key时，我们期待抛出AttributeError：

1
2
with self.assertRaises(AttributeError):
    value = d.empty

运行单元测试

一旦编写好单元测试，我们就可以运行单元测试。最简单的运行方式是在mydict_test.py的最后加上两行代码：

1
2
if __name__ == '__main__':
    unittest.main()

这样就可以把mydict_test.py当做正常的python脚本运行：

1
$ python mydict_test.py

另一种方法是在命令行通过参数-m unittest直接运行单元测试：

1
2
3
4
5
6
$ python -m unittest mydict_test
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

这是推荐的做法，因为这样可以一次批量运行很多单元测试，并且，有很多工具可以自动来运行这些单元测试。

例子：学生类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# TestStudent.py
import unittest

from student import Student

class TestStudent(unittest.TestCase): #编写一个测试类，从unittest.TestCase继承

    def test_80_to_100(self): #测试方法需要以test开头
        s1 = Student('Bart', 80)
        s2 = Student('Lisa', 100)
        self.assertEqual(s1.get_grade(), 'A')
        self.assertEqual(s2.get_grade(), 'A')

    def test_60_to_80(self): # 60-80，返回B
        s1 = Student('Bart', 60)
        s2 = Student('Lisa', 79)
        self.assertEqual(s1.get_grade(), 'B')
        self.assertEqual(s2.get_grade(), 'B')

    def test_0_to_60(self):
        s1 = Student('Bart', 0)
        s2 = Student('Lisa', 59)
        self.assertEqual(s1.get_grade(), 'C')
        self.assertEqual(s2.get_grade(), 'C')

    def test_invalid(self):
        s1 = Student('Bart', -1)
        s2 = Student('Lisa', 101)
        with self.assertRaises(ValueError):
            s1.get_grade()
        with self.assertRaises(ValueError):
            s2.get_grade()
1
2
3
4
5
6
7
8
9
10
11
class Student(object):
    def __init__(self, name, score):
        self.name = name
        self.score = score
    def get_grade(self):
        if self.score >= 60:
            return 'B'
        if self.score >= 80:
            return 'A'
        return 'C'

1
2
# 开始测试
python -m unittest TestStudent.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# 测试结果
(Test04) wjq@wjqdeMacBook-Pro test04_code % python -m unittest TestStudent.py
..FF
======================================================================
FAIL: test_80_to_100 (TestStudent.TestStudent)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wjq/Documents/GitHub/Test/Test04/test04_code/TestStudent.py", line 10, in test_80_to_100
    self.assertEqual(s1.get_grade(), 'A')
AssertionError: 'B' != 'A'
- B
+ A


======================================================================
FAIL: test_invalid (TestStudent.TestStudent)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wjq/Documents/GitHub/Test/Test04/test04_code/TestStudent.py", line 29, in test_invalid
    s1.get_grade()
AssertionError: ValueError not raised

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=2)




1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#  出错原因：分类比较混乱
# 改写
class Student(object):
    def __init__(self, name, score):
    	self.name = name
    	self.score = score
    def get_grade(self):
    	if self.score <0 or self.score>100:
    		raise ValueError
    	elif self.score >= 80:
    		return 'A'
    	elif self.score >= 60:
    		return 'B' 
    	return 'C'

需要注意空格和tab有区分，也会爆一些错误。

这个问题可以参考在sublime text3 设置里面加上两行代码解决。

1
2
3
4
// The number of spaces a tab is considered equal to
	"tab_size": 4,
	// Set to true to insert spaces when tab is pressed
	"translate_tabs_to_spaces": true,

装饰器

  • Allow you to tack on extra functionality to an already existing function.

  • 通过@进行定义

  • Function is also object

    遇到再研究，目前还不理解。

生成器

通过列表生成式，我们可以直接创建一个列表。但是，受到内存限制，列表容量肯定是有限的。而且，创建一个包含100万个元素的列表，不仅占用很大的存储空间，如果我们仅仅需要访问前面几个元素，那后面绝大多数元素占用的空间都白白浪费了。

所以，如果列表元素可以按照某种算法推算出来，那我们是否可以在循环的过程中不断推算出后续的元素呢？这样就不必创建完整的list，从而节省大量的空间。在Python中，这种一边循环一边计算的机制，称为生成器：generator。

要创建一个generator，有很多种方法。第一种方法很简单，只要把一个列表生成式的[]改成()，就创建了一个generator：

1
2
3
4
5
6
>>> L = [x * x for x in range(10)]
>>> L
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> g = (x * x for x in range(10))
>>> g
<generator object <genexpr> at 0x1022ef630>

Adavanced modules

  • Collections
  • OS modules and Datetime
  • Math and random
  • Python Debugger
  • Timeit
  • Regular Expression
  • Unzipping and zipping Modules

Google Colab