Python - 使用apply與自定義function建立dataframe的新欄位，常用於分組 - Applying function with multiple arguments to create a new pandas column

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

import pandas as pd
import numpy as np

df1 = pd.DataFrame()
number = [1,2,3,4,5]
sex = ['male','male','female','female','female']
df1['number'] = number
df1['sex'] = sex
df1['income'] = [500,1500,1300,2500,500]
print(df1)
print('-'*45)

def income_group(df):
    if df['income']>2000:
        return 'high'
    elif df['income']>1000:
        return 'medium'
    else:
        return 'low'

df1['group'] = df1.apply(income_group, axis=1)
print(df1)
print('-'*45)

Result:

   number     sex  income
0       1    male     500
1       2    male    1500
2       3  female    1300
3       4  female    2500
4       5  female     500
---------------------------------------------
   number     sex  income   group
0       1    male     500     low
1       2    male    1500  medium
2       3  female    1300  medium
3       4  female    2500    high
4       5  female     500     low
---------------------------------------------

​

Python - 使用條件來新增欄位，常用於將資料分組 - Pandas conditional creation of a dataframe column

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

import pandas as pd
import numpy as np

df1 = pd.DataFrame()
number = [1,2,3,4,5]
sex = ['male','male','female','female','female']
df1['number'] = number
df1['sex'] = sex
df1['income'] = [500,2000,500,2000,500]
print(df1)
print('='*45)

conditions = [
    (df1['sex']=='male') & (df1['income']>1000),
    (df1['sex']=='male') & (df1['income']<1000),
    (df1['sex']=='female') & (df1['income']>1000)
]                    

choices = ['male-high_income','male-low_income', 'female-high_income']
df1['group'] = np.select(conditions, choices, default='no_group')
print(df1)

Result:

   number     sex  income
0       1    male     500
1       2    male    2000
2       3  female     500
3       4  female    2000
4       5  female     500
=============================================
   number     sex  income               group
0       1    male     500     male-low_income
1       2    male    2000    male-high_income
2       3  female     500            no_group
3       4  female    2000  female-high_income
4       5  female     500            no_group

​

Python - pandas dataframe的append處理後保留欄位名稱的順序 - How to control column order when appending to a DataFrame

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

import pandas as pd

df1 = pd.DataFrame()
number = [1,2,3,4,5]
sex = ['male','female','female','female','male']
df1['number'] = number
df1['sex'] = sex
df1['b'] = [1,2,3,4,5]

df2 = pd.DataFrame()
number = [1,2,3,4,5]
sex = ['male','female','female','female','male']
df2['number'] = number
df2['sex'] = sex
df2['a'] = [1,2,3,4,5]
print('df1')
print(df1)
print('-'*45)
print('df2')
print(df2)
print('-'*45)

print('未處理欄位名稱，欄位名稱重新排序')
df3 = df1.append(df2,ignore_index =True)
print(df3)
print('-'*45)

print('處理後，欄位名稱會以df1為準，新的欄位依序加入')

columns_all = list(df1.columns)
columns_new = list(df2.columns)

for ele in columns_new:
    if ele not in columns_all:
        columns_all.append(ele)

df3 = df1.append(df2,ignore_index =True)
df3 = df3[columns_all]
print(df3)
print('-'*45)

Result:

df1
   number     sex  b
0       1    male  1
1       2  female  2
2       3  female  3
3       4  female  4
4       5    male  5
---------------------------------------------
df2
   number     sex  a
0       1    male  1
1       2  female  2
2       3  female  3
3       4  female  4
4       5    male  5
---------------------------------------------
未處理欄位名稱，欄位名稱重新排序
     a    b  number     sex
0  NaN  1.0       1    male
1  NaN  2.0       2  female
2  NaN  3.0       3  female
3  NaN  4.0       4  female
4  NaN  5.0       5    male
5  1.0  NaN       1    male
6  2.0  NaN       2  female
7  3.0  NaN       3  female
8  4.0  NaN       4  female
9  5.0  NaN       5    male
---------------------------------------------
處理後，欄位名稱會以df1為準，新的欄位依序加入
   number     sex    b    a
0       1    male  1.0  NaN
1       2  female  2.0  NaN
2       3  female  3.0  NaN
3       4  female  4.0  NaN
4       5    male  5.0  NaN
5       1    male  NaN  1.0
6       2  female  NaN  2.0
7       3  female  NaN  3.0
8       4  female  NaN  4.0
9       5    male  NaN  5.0
---------------------------------------------

​

Python - 如何使用對list使用去重複、交集、聯集 - How to use unique, intersection, union lists in Python

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

def unique(a):
    return list(set(a))

def intersect(a, b):
    return list(set(a) & set(b))

def union(a, b):
    return list(set(a) | set(b))

a = [1,2,3,4,4]
b = [2,3,5]
print( unique(a) )
print( intersect(a, b) )
print( union(a, b) )

Result:

[1, 2, 3, 4]
[2, 3]
[1, 2, 3, 4, 5]

​

Python - 在 Pandas DataFrame 中去除重複的row - How to drop duplicate rows in Python Pandas DataFrame- Stack Overflow

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

先建立資料

Code:

number = [1,2,3,4,5]
sex = ['male','female','female','female','male']
df_new = pd.DataFrame()
df_new['number'] = number
df_new['sex'] = sex
print(df_new)

Result:

   number     sex
0       1    male
1       2  female
2       3  female
3       4  female
4       5    male

去重複

Code:

df_new.drop_duplicates(['sex'])

Result:

   number     sex
0       1    male
1       2  female

​

Python - 將字串轉為字典 - Convert a String representation of a Dictionary to a dictionary?

Information:

System version : Windows 10 64-bit
Python version : Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

import ast
str1 = "{'a' : 1, 'b' : 2}"
dict1 = ast.literal_eval(str1)
print(dict1)

Result:

{'a': 1, 'b': 2}

​

Python - 在Python上使用交集、聯集、差集 - How to use intersection,difference,union in Python

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

A = set([1,2,3,4])
B = set([2,3,5])

# 交集
set_list = [A,B]
intersection_A_B = set.intersection(*set_list)
# 差集
difference_A_B = A.difference(B)
difference_B_A = B.difference(A)
# 聯集
union_A_B = A.union(B)

print('交集')
print('intersection_A_B :',intersection_A_B)
print('差集')
print('difference_A_B :',difference_A_B)
print('difference_B_A :',difference_B_A)
print('聯集')
print('union_A_B :',union_A_B)

Result:

交集
intersection_A_B : {2, 3}
差集
difference_A_B : {1, 4}
difference_B_A : {5}
聯集
union_A_B : {1, 2, 3, 4, 5}

​

Python - pandasql - 如何在Python中使用sql語法操作dataframe -Querying a dataframe with SQL

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

先建立資料

Code:

number = [1,2,3,4,5]
sex = ['male','female','female','female','male']
df_new = pd.DataFrame()
df_new['number'] = number
df_new['sex'] = sex
print(df_new)

Result:

   number     sex
0       1    male
1       2  female
2       3  female
3       4  female
4       5    male

使用sql語法來操作dataframe

Code:

from pandasql import sqldf
q = 'SELECT * FROM df_new  where sex = "male";'
df_one= sqldf(q)
print(df_one)

Result:

   number   sex
0       1  male
1       5  male

​

Python - 在Pandas DataFrame中缺失資料的處理-以上一個row的值來填補 - How to replace NaN(None,Null) by preceding values in pandas DataFrame?

Information：

System version ： Windows 10 64-bit
Python version ： Python 3.6.0 :: Anaconda 4.3.1 (64-bit)

Code:

df1 = pd.DataFrame([[3, 1, 8], [None, 5, None], [None, None, 9]])
df1_fillna = df1.fillna(method='ffill')
print('df1')
print(df1)
print('='*20)
print('df1_fillna')
print(df1_fillna)

Result:

df1
     0    1    2
0  3.0  1.0  8.0
1  NaN  5.0  NaN
2  NaN  NaN  9.0
====================
df1_fillna
     0    1    2
0  3.0  1.0  8.0
1  3.0  5.0  8.0
2  3.0  5.0  9.0

​