跳转至

NumPy 数组

创建数组

Python
a = np.array([[1,2],
              [3,4]])

创建随机数组

Python
np.random.rand(10)
# array([ 0.56911206,  0.99777291,  0.18943144,  0.19387287,  0.75090637,
#         0.18692814,  0.69804514,  0.48808425,  0.79440667,  0.66959075])
  • 上面的代码生成 [0.0, 1.0) 之间的随机浮点数,当没有参数时,返回一个随机浮点数,当有一个参数时,返回该参数长度大小的一维随机浮点数数组。
Python
np.random.randn(10)
# array([-1.6765704 ,  0.66361856,  0.04029481,  1.19965741, -0.57514593,
#        -0.79603968,  1.52261545, -2.17401814,  0.86671727, -1.17945975])
  • 上面的代码返回一个随机数组,具有标准正态分布。
Python
np.random.randint(10,size=10)
# array([4, 1, 4, 3, 8, 2, 8, 5, 8, 9])
  • 上面的代码返回随机的整数,位于半开区间 [low, high)。

生成相同元素组成的数组

Python
np.full((2, 2), 10)

对每一个元素应用同一个函数

方法一:np.vectorize()

要对 NumPy 数组中的每个元素应用一个函数,可以使用 NumPy 的 np.vectorize() 函数。np.vectorize() 函数可以将一个普通的 Python 函数转化为适用于 NumPy 数组的函数。

下面是一个示例,演示如何对 NumPy 数组中的每个元素应用一个函数:

Python
import numpy as np

# 定义一个函数,将元素加倍
def double(x):
    return 2 * x

# 创建一个 NumPy 数组
arr = np.array([1, 2, 3, 4, 5])

# 使用 np.vectorize() 函数将函数转化为适用于 NumPy 数组的函数
vectorized_double = np.vectorize(double)

# 应用函数到数组的每个元素
result = vectorized_double(arr)

print(result)

输出结果为:

Text Only
[ 2  4  6  8 10]

在上面的示例中,我们定义了一个名为 double() 的函数,它将输入的值加倍。然后,我们使用 np.vectorize() 函数将该函数转化为适用于 NumPy 数组的函数 vectorized_double。最后,我们将 vectorized_double 函数应用到 NumPy 数组 arr 的每个元素上,得到了每个元素加倍的结果。

np.vectorize() 并没有优化性能,其本质仍是 for 循环

官方文档:The vectorize function is provided primarily for convenience, not for performance. The implementation is essentially a for loop.

方法二:np.frompyfunc()

经测试,np.frompyfunc() 耗时仅为 np.vectorize() 的约 30%。在同样的运算量下,前者需 1 秒,后者需 3 秒。

Python
import numpy as np

# 定义一个函数,将元素加倍
def double(x):
    return 2 * x

# 使用 numpy.frompyfunc() 将函数转化为通用函数
ufunc_double = np.frompyfunc(double, 1, 1)

# 创建一个 NumPy 数组
arr = np.array([1, 2, 3, 4, 5])

# 应用通用函数到数组的每个元素
result = ufunc_double(arr)

# 将结果转换为 NumPy 数组
result = np.array(result)

print(result)

矩阵求和:两个矩阵按元素求和,若有空值则相加得 0

Python
c = np.nansum(np.stack((a,b)), axis=0)

参考:https://stackoverflow.com/a/50642601/

矩阵除法:两个矩阵按元素求商,若遇到分母为 0 则将结果设为 0

Python
a = np.array([-1, 0, 1, 2, 3], dtype=float)
b = np.array([ 0, 0, 0, 2, 2], dtype=float)
c = np.divide(a, b, out=np.zeros_like(a), where=b!=0)
# >>> print(c)
# [ 0.   0.   0.   1.   1.5]

参考:https://stackoverflow.com/a/37977222/

在一个已排序的数组中,找到某个元素应该插入的位置

参考:https://numpy.org/doc/stable/reference/generated/numpy.searchsorted.html

Python
import numpy as np

a = np.array([1, 2, 4, 4, 4, 6])
v = np.array([0, 1, 2, 3, 4, 5, 6, 7])

print(np.searchsorted(a, v))
# [0 0 1 2 2 5 5 6]

print(np.searchsorted(a, v, side='right'))
# [0 1 2 2 5 5 6 6]

读取txt文件

参考:https://numpy.org/doc/stable/reference/generated/numpy.loadtxt.html

Python
np.loadtxt("test.txt", skiprows=3, delimiter=",", dtype=np.int32)

导出txt文件

参考:https://numpy.org/doc/stable/reference/generated/numpy.savetxt.html

Python
np.savetxt(
    "test.txt",
    my_array,
    fmt="%d",
    delimiter=",",
    header="col_name_1,col_name_2,col_name_3",
    comments='',
)

comments 可以避免第一行的列名出现 #。参考:https://stackoverflow.com/a/17361181/

向量内积

Python
import numpy as np

# 创建两个数组
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

# 计算内积
dot_product = np.dot(a, b)

print(dot_product)

image-20230220225628610

向量外积

一个n*1 的列向量乘以一个1*n 的行向量,得到的是一个n*n 的矩阵。如果用a*bnp.dot(a, b),得到的都是两个列向量的内积,即一个标量,并不是我们想要的矩阵。

可以用np.outer实现。

Python
a = np.array([1,2]).reshape(2,1)
b = np.array([3,4]).reshape(1,2)
np.outer(a,b)

image-20221015170934229

计算相关系数矩阵

如果只需要计算 Pearson 相关系数,可以用 numpy.corrcoef

注意 rowvar 参数指定了:一行为一个观测值或一列为一个观测值。

如果需要计算 Spearman 秩相关系数,需要用 scipy.stats.spearmanr

注意 axis 参数指定了:一行为一个观测值或一列为一个观测值。

将一个上三角形矩阵转换为对称矩阵

Python
test =np.nansum(np.stack((test, test.T)),axis=0)

将矩阵的对角线元素全部设为 1

Python
np.fill_diagonal(matrix, 1)

将两个矩阵拼接成广义上的对角矩阵

可以使用 NumPy 库的block函数将两个矩阵拼接成广义上的对角矩阵。具体实现如下:

Python
import numpy as np

# 假设有两个 3x3 的矩阵 A 和 B
A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
B = np.array([[10, 11, 12], [13, 14, 15], [16, 17, 18]])

# 使用 block 函数将 A 和 B 拼接成广义上的对角矩阵 C
C = np.block([[A, np.zeros((3, 3))], [np.zeros((3, 3)), B]])

print(C)

输出结果为:

Text Only
[[ 1.  2.  3.  0.  0.  0.]
 [ 4.  5.  6.  0.  0.  0.]
 [ 7.  8.  9.  0.  0.  0.]
 [ 0.  0.  0. 10. 11. 12.]
 [ 0.  0.  0. 13. 14. 15.]
 [ 0.  0.  0. 16. 17. 18.]]

其中,np.zeros((3, 3))表示一个 3x3 的全零矩阵,np.block函数将四个子矩阵按照指定位置拼接起来。

拼接多个切片

Python
np.r_[0:5, 10:15]

参考:

https://numpy.org/doc/stable/reference/generated/numpy.r_.html

https://stackoverflow.com/a/30597960/

双冒号切片

基本形式:[开始:结束:步长]

例:步长 step=n,代表从 start 开始(包括 start)每隔 step 间隔,取一个数,一直到结尾 end(不包括 end)。

Python
range(20)[::3]
[0,3,6,9,12,15,18]

例:当 step 等于负数的时候,从右向左取数。

Python
range(10)[::-1]
[9,8,7,6,5,4,3,2,1,0]

range(10)[::-2]
[9,7,5,3,1]

对数组中的元素计数

Python
a = np.array([0, 3, 0, 1, 0, 1, 2, 1, 0, 0, 0, 0, 1, 3, 4])
unique, counts = np.unique(a, return_counts=True)

>>> dict(zip(unique, counts))
{0: 7, 1: 4, 2: 1, 3: 2, 4: 1}

堆叠成新数组

hstack是 horizontal,左右水平堆叠。

image-20230117142319140

vstack是 vertical,上下垂直堆叠。

Python
import numpy as np
a=[1, 2, 3]
b=[4, 5, 6]
c=[7, 8, 9]
print(np.vstack((a,b,c)))

image-20230117142322373

计算非空值的个数

Python
arr = np.array([np.nan, 2, 3, 4, 5])

(~np.isnan(arr)).sum()
# 4

排序后的索引值

返回从小到大排序后的索引值。

Python
a = np.array([2,1,4,3])
np.argsort(a)

image-20230117145058544

找出每行中最小或最大的 k 个数

Python
np.argpartition(a, kth=k)

image-20230313202248818

用一个数组的元素作为索引,取出另一个数组中的元素

我们需要从 a 中提取元素。b 是另一个数组,其中的值就是要从 a 中取出的元素的索引。

例如,若 b[1, 1] = 5,这意味着我需要在新的数组中的第二行第二列显示a[1, 5]的数值。

Python
new_arr = np.zeros_like(b)
for i in range(len(b)):
    for j in range(len(b[i])): new_arr[i][j] = a[i][b[i][j]]

效果示例见“找出每行中最小或最大的 k 个数”。

生成等差数组

Python
np.linspace(start, stop, num)

默认左右都是闭区间。

可以指定endpoint参数(默认为True)来设置是否包含最后一个数字:

Python
np.linspace(start, stop, num, endpoint=False)

将数据分成多个 bins(桶、箱)

Python
a = np.random.randint(10,size=10)
bins = np.histogram(a)[0] # 默认基于最小值和最大值分成 10 个 bins

np.histogram 的用法:

Python
numpy.histogram(a, bins=10, range=None, density=None, weights=None)

https://numpy.org/doc/stable/reference/generated/numpy.histogram.html

image-20230209005552664

计算熵

np.histogram对数据进行分桶后,可以计算熵以衡量数据的分散程度:

Python
entropy = 0
for i in bins:
    p = i / sum(bins)
    entropy += -p * np.log(p)

交叉熵损失函数介绍了熵的定义。

防止 Numpy 输出宽矩阵时自动换行

在 Numpy 中,打印矩阵时默认会自动换行,以适应输出窗口的尺寸。如果想要防止自动换行,可以设置 Numpy 的打印选项。

具体来说,可以使用以下代码使 Numpy 不自动换行:

Python
import numpy as np

np.set_printoptions(linewidth=np.inf)

其中,np.set_printoptions()是 Numpy 的打印选项函数,可以用来设置打印矩阵时的一些参数。linewidth参数指定了输出的每行字符数的最大值,将其设置为np.inf意味着没有限制,可以输出任意长的行。

注意,使用np.set_printoptions()设置的打印选项会对整个 Numpy 库生效,因此如果要恢复默认的行宽限制,需要重新调用一次np.set_printoptions(),并重新指定一个较小的行宽限制。

NumPy 100 道练习题

以下题目来自:https://github.com/rougier/numpy-100。通读一遍后,有些函数和用法是很值得学习的。

1. Import the NumPy package under the name np (★☆☆)

hint: import … as

Python
import numpy as np

2. Print the NumPy version and the configuration (★☆☆)

hint: np.__version__, np.show_config)

Python
print(np.__version__)
np.show_config()

3. Create a null vector of size 10 (★☆☆)

hint: np.zeros

Python
Z = np.zeros(10)
print(Z)

4. How to find the memory size of any array (★☆☆)

hint: size, itemsize

Python
Z = np.zeros((10,10))
print("%d bytes" % (Z.size * Z.itemsize))

或直接使用

Python
Z = np.zeros((10,10))
print("%d bytes" % (Z.nbytes))

参考:

https://numpy.org/doc/stable/reference/generated/numpy.ndarray.itemsize.html

https://stackoverflow.com/a/11784399/

5. How to get the documentation of the NumPy add function from the command line? (★☆☆)

hint: np.info

Python
%run `python -c "import numpy; numpy.info(numpy.add)"`

Note

np.info()会获取文档,更简洁易读。而help()会获取源代码。

6. Create a null vector of size 10 but the fifth value which is 1 (★☆☆)

hint: array[4]

Python
Z = np.zeros(10)
Z[4] = 1
print(Z)

7. Create a vector with values ranging from 10 to 49 (★☆☆)

hint: arange

Python
Z = np.arange(10,50)
print(Z)

8. Reverse a vector (first element becomes last) (★☆☆)

hint: array[::-1]

Python
Z = np.arange(50)
Z = Z[::-1]
print(Z)

9. Create a 3x3 matrix with values ranging from 0 to 8 (★☆☆)

hint: reshape

Python
Z = np.arange(9).reshape(3, 3)
print(Z)

10. Find indices of non-zero elements from [1,2,0,0,4,0] (★☆☆) 非零元素的索引

hint: np.nonzero

Python
nz = np.nonzero([1,2,0,0,4,0])
print(nz)

11. Create a 3x3 identity matrix (★☆☆) 对角矩阵

hint: np.eye

Python
Z = np.eye(3)
print(Z)

12. Create a 3x3x3 array with random values (★☆☆) 创建随机矩阵

hint: np.random.random

Python
Z = np.random.random((3,3,3))
print(Z)

13. Create a 10x10 array with random values and find the minimum and maximum values (★☆☆)

hint: min, max

Python
Z = np.random.random((10,10))
Zmin, Zmax = Z.min(), Z.max()
print(Zmin, Zmax)

14. Create a random vector of size 30 and find the mean value (★☆☆)

hint: mean

Python
Z = np.random.random(30)
m = Z.mean()
print(m)

15. Create a 2d array with 1 on the border and 0 inside (★☆☆)

hint: array[1:-1, 1:-1]

Python
Z = np.ones((10,10))
Z[1:-1,1:-1] = 0
print(Z)

16. How to add a border (filled with 0's) around an existing array? (★☆☆) 在矩阵周围加上一圈

hint: np.pad

Python
Z = np.ones((5,5))
Z = np.pad(Z, pad_width=1, mode='constant', constant_values=0)
print(Z)

# Using fancy indexing
Z[:, [0, -1]] = 0
Z[[0, -1], :] = 0
print(Z)
Python
np.random.seed(1)
x = np.random.randn(3,3)
np.pad(x, ((1,2), (1,1)), 'constant', constant_values = (0,0))

image-20230327232047354

17. What is the result of the following expression? (★☆☆) 空值的相关判断

Python
0 * np.nan
np.nan == np.nan
np.inf > np.nan
np.nan - np.nan
np.nan in set([np.nan])
0.3 == 3 * 0.1

hint: NaN = not a number, inf = infinity

Python
print(0 * np.nan)
print(np.nan == np.nan)
print(np.inf > np.nan)
print(np.nan - np.nan)
print(np.nan in set([np.nan]))
print(0.3 == 3 * 0.1)

image-20230222165538049

0.3 == 3 * 0.1 是由于浮点数运算的精度引起的。

image-20230222165833241

image-20230222165855884

18. Create a 5x5 matrix with values 1,2,3,4 just below the diagonal (★☆☆)

hint: np.diag

Python
Z = np.diag(1+np.arange(4),k=-1)
print(Z)

k=-1 代表了向下一格的对角线。

image-20230222170111250

19. Create a 8x8 matrix and fill it with a checkerboard pattern (★☆☆)

hint: array[::2]

Python
Z = np.zeros((8,8),dtype=int)
Z[1::2,::2] = 1
Z[::2,1::2] = 1
print(Z)

# Alternative solution: Using reshaping
arr = np.ones(64,dtype=int)
arr[::2]=0
arr = arr.reshape((8,8))
print(arr)

20. Consider a (6,7,8) shape array, what is the index (x,y,z) of the 100th element? (★☆☆) 找到第几个元素的索引

hint: np.unravel_index

Python
print(np.unravel_index(99,(6,7,8)))

21. Create a checkerboard 8x8 matrix using the tile function (★☆☆) 将一个矩阵在多个维度上重复几遍

hint: np.tile

Python
Z = np.tile( np.array([[0,1],[1,0]]), (4,4))
print(Z)

image-20230222170602602

22. Normalize a 5x5 random matrix (★☆☆)

hint: (x -mean)/std

Python
Z = np.random.random((5,5))
Z = (Z - np.mean (Z)) / (np.std (Z))
print(Z)

23. Create a custom dtype that describes a color as four unsigned bytes (RGBA) (★☆☆)

hint: np.dtype

Python
color = np.dtype([("r", np.ubyte),
                  ("g", np.ubyte),
                  ("b", np.ubyte),
                  ("a", np.ubyte)])

24. Multiply a 5x3 matrix by a 3x2 matrix (real matrix product) (★☆☆)

hint:

Python
Z = np.dot(np.ones((5,3)), np.ones((3,2)))
print(Z)

# Alternative solution, in Python 3.5 and above
Z = np.ones((5,3)) @ np.ones((3,2))
print(Z)

25. Given a 1D array, negate all elements which are between 3 and 8, in place. (★☆☆)

hint: >, <

Python
# Author: Evgeni Burovski

Z = np.arange(11)
Z[(3 < Z) & (Z < 8)] *= -1
print(Z)

26. What is the output of the following script? (★☆☆)

Python
# Author: Jake VanderPlas

print(sum(range(5),-1))
from numpy import *
print(sum(range(5),-1))

hint: np.sum

image-20230222225856020

Python
# Author: Jake VanderPlas

print(sum(range(5),-1))
from numpy import *
print(sum(range(5),-1))
Python
Z**Z
2 << Z >> 2
Z <- Z
1j*Z
Z/1/1
Z<Z>Z

No hints provided...

Python
Z**Z
2 << Z >> 2
Z <- Z
1j*Z
Z/1/1
Z<Z>Z

28. What are the result of the following expressions? (★☆☆)

Python
np.array(0) / np.array(0)
np.array(0) // np.array(0)
np.array([np.nan]).astype(int).astype(float)

No hints provided...

Python
print(np.array(0) / np.array(0))
print(np.array(0) // np.array(0))
print(np.array([np.nan]).astype(int).astype(float))

29. How to round away from zero a float array ? (★☆☆) 改变数组的正负号

hint: np.uniform, np.copysign, np.ceil, np.abs, np.where

np.copysign

Text Only
copysign(x1, x2, /, out=None, *, where=True, casting='same_kind', order='K', dtype=None, subok=True[, signature, extobj])

Change the sign of x1 to that of x2, element-wise.

If `x2` is a scalar, its sign will be copied to all elements of `x1`.
Python
# Author: Charles R Harris

Z = np.random.uniform(-10,+10,10)
print(np.copysign(np.ceil(np.abs(Z)), Z))

# More readable but less efficient
print(np.where(Z>0, np.ceil(Z), np.floor(Z)))

30. How to find common values between two arrays? (★☆☆) 取交集

hint: np.intersect1d

Python
Z1 = np.random.randint(0,10,10)
Z2 = np.random.randint(0,10,10)
print(np.intersect1d(Z1,Z2))

hint: np.seterr, np.errstate

Python
# Suicide mode on
defaults = np.seterr(all="ignore")
Z = np.ones(1) / 0

# Back to sanity
_ = np.seterr(**defaults)

# Equivalently with a context manager
with np.errstate(all="ignore"):
    np.arange(3) / 0

32. Is the following expressions true? (★☆☆)

Python
np.sqrt(-1) == np.emath.sqrt(-1)

hint: imaginary number

Python
np.sqrt(-1) == np.emath.sqrt(-1)

33. How to get the dates of yesterday, today and tomorrow? (★☆☆) datetime 格式

hint: np.datetime64, np.timedelta64

Python
yesterday = np.datetime64('today') - np.timedelta64(1)
today     = np.datetime64('today')
tomorrow  = np.datetime64('today') + np.timedelta64(1)

34. How to get all the dates corresponding to the month of July 2016? (★★☆) 时间区间里的所有天数

hint: np.arange(dtype=datetime64['D'])

Python
Z = np.arange('2016-07', '2016-08', dtype='datetime64[D]')
print(Z)

35. How to compute ((A+B)*(-A/2)) in place (without copy)? (★★☆) 使用out=直接赋值给变量

hint: np.add(out=), np.negative(out=), np.multiply(out=), np.divide(out=)

Python
A = np.ones(3)*1
B = np.ones(3)*2
np.add(A,B,out=B)
np.divide(A,2,out=A)
np.negative(A,out=A)
np.multiply(A,B,out=A)

36. Extract the integer part of a random array of positive numbers using 4 different methods (★★☆) np.trunc 取离 0 最近的整数

hint: %, np.floor, astype, np.trunc

Python
Z = np.random.uniform(0,10,10)

print(Z - Z%1)
print(Z // 1)
print(np.floor(Z))
print(Z.astype(int))
print(np.trunc(Z))

37. Create a 5x5 matrix with row values ranging from 0 to 4 (★★☆)

hint: np.arange

Python
Z = np.zeros((5,5))
Z += np.arange(5)
print(Z)

# without broadcasting
Z = np.tile(np.arange(0, 5), (5,1))
print(Z)

38. Consider a generator function that generates 10 integers and use it to build an array (★☆☆)

hint: np.fromiter

Python
def generate():
    for x in range(10):
        yield x
Z = np.fromiter(generate(),dtype=float,count=-1)
print(Z)

39. Create a vector of size 10 with values ranging from 0 to 1, both excluded (★★☆)

hint: np.linspace

Python
Z = np.linspace(0,1,11,endpoint=False)[1:]
print(Z)

40. Create a random vector of size 10 and sort it (★★☆)

hint: sort

Python
Z = np.random.random(10)
Z.sort()
print(Z)

41. How to sum a small array faster than np.sum? (★★☆)

hint: np.add.reduce

Python
# Author: Evgeni Burovski

Z = np.arange(10)
np.add.reduce(Z)

42. Consider two random array A and B, check if they are equal (★★☆) 判断两个数组是否相等

np.allclose 可以允许存在一定的偏差,np.array_equal 必须完全一样。

hint: np.allclose, np.array_equal

Python
A = np.random.randint(0,2,5)
B = np.random.randint(0,2,5)

# Assuming identical shape of the arrays and a tolerance for the comparison of values
equal = np.allclose(A,B)
print(equal)

# Checking both the shape and the element values, no tolerance (values have to be exactly equal)
equal = np.array_equal(A,B)
print(equal)

43. Make an array immutable (read-only) (★★☆) 使数组变得不可更改(只读)

hint: flags.writeable

Python
Z = np.zeros(10)
Z.flags.writeable = False
Z[0] = 1

44. Consider a random 10x2 matrix representing cartesian coordinates, convert them to polar coordinates (★★☆)

hint: np.sqrt, np.arctan2

Python
Z = np.random.random((10,2))
X,Y = Z[:,0], Z[:,1]
R = np.sqrt(X**2+Y**2)
T = np.arctan2(Y,X)
print(R)
print(T)

45. Create random vector of size 10 and replace the maximum value by 0 (★★☆)

hint: argmax

Python
Z = np.random.random(10)
Z[Z.argmax()] = 0
print(Z)

46. Create a structured array with x and y coordinates covering the [0,1]x[0,1] area (★★☆) 生成网格数据,使用np.zeros时可以给定列名

hint: np.meshgrid

Python
Z = np.zeros((5,5), [('x',float),('y',float)])
Z['x'], Z['y'] = np.meshgrid(np.linspace(0,1,5),
                             np.linspace(0,1,5))
print(Z)

47. Given two arrays, X and Y, construct the Cauchy matrix C (Cij =1/(xi - yj)) (★★☆)

hint: np.subtract.outer

Python
# Author: Evgeni Burovski

X = np.arange(8)
Y = X + 0.5
C = 1.0 / np.subtract.outer(X, Y)
print(np.linalg.det(C))

48. Print the minimum and maximum representable value for each NumPy scalar type (★★☆) 各种dtype能表示的数据范围

hint: np.iinfo, np.finfo, eps

Python
for dtype in [np.int8, np.int32, np.int64]:
   print(np.iinfo(dtype).min)
   print(np.iinfo(dtype).max)
for dtype in [np.float32, np.float64]:
   print(np.finfo(dtype).min)
   print(np.finfo(dtype).max)
   print(np.finfo(dtype).eps)

49. How to print all the values of an array? (★★☆) 打印所有行

hint: np.set_printoptions

Python
np.set_printoptions(threshold=float("inf"))
Z = np.zeros((40,40))
print(Z)

50. How to find the closest value (to a given scalar) in a vector? (★★☆)

hint: argmin

Python
Z = np.arange(100)
v = np.random.uniform(0,100)
index = (np.abs(Z-v)).argmin()
print(Z[index])

51. Create a structured array representing a position (x,y) and a color (r,g,b) (★★☆) 创建自定义结构的数组

hint: dtype

Python
Z = np.zeros(10, [ ('position', [ ('x', float, 1),
                                  ('y', float, 1)]),
                   ('color',    [ ('r', float, 1),
                                  ('g', float, 1),
                                  ('b', float, 1)])])
print(Z)

52. Consider a random vector with shape (100,2) representing coordinates, find point by point distances (★★☆)

hint: np.atleast_2d, T, np.sqrt

Python
Z = np.random.random((10,2))
X,Y = np.atleast_2d(Z[:,0], Z[:,1])
D = np.sqrt( (X-X.T)**2 + (Y-Y.T)**2)
print(D)

# Much faster with scipy
import scipy
# Thanks Gavin Heverly-Coulson (#issue 1)
import scipy.spatial

Z = np.random.random((10,2))
D = scipy.spatial.distance.cdist(Z,Z)
print(D)

53. How to convert a float (32 bits) array into an integer (32 bits) in place?

hint: view and [:] =

Python
# Thanks Vikas (https://stackoverflow.com/a/10622758/5989906)
# & unutbu (https://stackoverflow.com/a/4396247/5989906)
Z = (np.random.rand(10)*100).astype(np.float32)
Y = Z.view(np.int32)
Y[:] = Z
print(Y)

54. How to read the following file? (★★☆) 从字符串中读取数组

Text Only
1, 2, 3, 4, 5
6,  ,  , 7, 8
 ,  , 9,10,11

hint: np.genfromtxt

Python
from io import StringIO

# Fake file
s = StringIO('''1, 2, 3, 4, 5

                6,  ,  , 7, 8

                 ,  , 9,10,11
''')
Z = np.genfromtxt(s, delimiter=",", dtype=np.int)
print(Z)

55. What is the equivalent of enumerate for NumPy arrays? (★★☆) 遍历输出每个元素

hint: np.ndenumerate, np.ndindex

Python
Z = np.arange(9).reshape(3,3)
for index, value in np.ndenumerate(Z):
    print(index, value)
for index in np.ndindex(Z.shape):
    print(index, Z[index])

56. Generate a generic 2D Gaussian-like array (★★☆) 生成二维高斯分布的概率密度

hint: np.meshgrid, np.exp

Python
X, Y = np.meshgrid(np.linspace(-1,1,10), np.linspace(-1,1,10))
D = np.sqrt(X*X+Y*Y)
sigma, mu = 1.0, 0.0
G = np.exp(-( (D-mu)**2 / ( 2.0 * sigma**2 ) ) )
print(G)

57. How to randomly place p elements in a 2D array? (★★☆) 替换某些元素,类似于update

hint: np.put, np.random.choice

Python
# Author: Divakar

n = 10
p = 3
Z = np.zeros((n,n))
np.put(Z, np.random.choice(range(n*n), p, replace=False),1)
print(Z)

58. Subtract the mean of each row of a matrix (★★☆) 减去每行的均值

hint: mean(axis=,keepdims=)

Python
# Author: Warren Weckesser

X = np.random.rand(5, 10)

# Recent versions of NumPy
Y = X - X.mean(axis=1, keepdims=True)

# Older versions of NumPy
Y = X - X.mean(axis=1).reshape(-1, 1)

print(Y)

59. How to sort an array by the nth column? (★★☆) 按列排序

hint: argsort

Python
# Author: Steve Tjoa

Z = np.random.randint(0,10,(3,3))
print(Z)
print(Z[Z[:,1].argsort()])

60. How to tell if a given 2D array has null columns? (★★☆) 判断空值

hint: any, ~

Python
# Author: Warren Weckesser

# null : 0
Z = np.random.randint(0,3,(3,10))
print((~Z.any(axis=0)).any())

# null : np.nan
Z=np.array([
    [0,1,np.nan],
    [1,2,np.nan],
    [4,5,np.nan]
])
print(np.isnan(Z).all(axis=0))

61. Find the nearest value from a given value in an array (★★☆)

hint: np.abs, argmin, flat

Python
Z = np.random.uniform(0,1,10)
z = 0.5
m = Z.flat[np.abs(Z - z).argmin()]
print(m)

62. Considering two arrays with shape (1,3) and (3,1), how to compute their sum using an iterator? (★★☆) 列向量与行向量求和,生成数组

hint: np.nditer

Python
A = np.arange(3).reshape(3,1)
B = np.arange(3).reshape(1,3)
it = np.nditer([A,B,None])
for x,y,z in it: z[...] = x + y
print(it.operands[2])

63. Create an array class that has a name attribute (★★☆)

hint: class method

Python
class NamedArray(np.ndarray):
    def __new__(cls, array, name="no name"):
        obj = np.asarray(array).view(cls)
        obj.name = name
        return obj
    def __array_finalize__(self, obj):
        if obj is None: return
        self.name = getattr(obj, 'name', "no name")

Z = NamedArray(np.arange(10), "range_10")
print (Z.name)

64. Consider a given vector, how to add 1 to each element indexed by a second vector (be careful with repeated indices)? (★★★)

hint: np.bincount | np.add.at

Python
# Author: Brett Olsen

Z = np.ones(10)
I = np.random.randint(0,len(Z),20)
Z += np.bincount(I, minlength=len(Z))
print(Z)

# Another solution
# Author: Bartosz Telenczuk
np.add.at(Z, I, 1)
print(Z)

65. How to accumulate elements of a vector (X) to an array (F) based on an index list (I)? (★★★) 分桶统计,可为每个数据提供频数(权重)

hint: np.bincount

Python
# Author: Alan G Isaac

X = [1,2,3,4,5,6]
I = [1,3,9,3,4,1]
F = np.bincount(I,X)
print(F)

66. Considering a (w,h,3) image of (dtype=ubyte), compute the number of unique colors (★★☆)

hint: np.unique

Python
# Author: Fisher Wang

w, h = 256, 256
I = np.random.randint(0, 4, (h, w, 3)).astype(np.ubyte)
colors = np.unique(I.reshape(-1, 3), axis=0)
n = len(colors)
print(n)

# Faster version
# Author: Mark Setchell
# https://stackoverflow.com/a/59671950/2836621

w, h = 256, 256
I = np.random.randint(0,4,(h,w,3), dtype=np.uint8)

# View each pixel as a single 24-bit integer, rather than three 8-bit bytes
I24 = np.dot(I.astype(np.uint32),[1,256,65536])

# Count unique colours
n = len(np.unique(I24))
print(n)

67. Considering a four dimensions array, how to get sum over the last two axis at once? (★★★)

hint: sum(axis=(-2,-1))

Python
A = np.random.randint(0,10,(3,4,3,4))
# solution by passing a tuple of axes (introduced in NumPy 1.7.0)
sum = A.sum(axis=(-2,-1))
print(sum)
# solution by flattening the last two dimensions into one
# (useful for functions that don't accept tuples for axis argument)
sum = A.reshape(A.shape[:-2] + (-1,)).sum(axis=-1)
print(sum)

68. Considering a one-dimensional vector D, how to compute means of subsets of D using a vector S of same size describing subset indices? (★★★)

hint: np.bincount

Python
# Author: Jaime Fernández del Río

D = np.random.uniform(0,1,100)
S = np.random.randint(0,10,100)
D_sums = np.bincount(S, weights=D)
D_counts = np.bincount(S)
D_means = D_sums / D_counts
print(D_means)

# Pandas solution as a reference due to more intuitive code
import pandas as pd
print(pd.Series(D).groupby(S).mean())

69. How to get the diagonal of a dot product? (★★★)

hint: np.diag

Python
# Author: Mathieu Blondel

A = np.random.uniform(0,1,(5,5))
B = np.random.uniform(0,1,(5,5))

# Slow version
np.diag(np.dot(A, B))

# Fast version
np.sum(A * B.T, axis=1)

# Faster version
np.einsum("ij,ji->i", A, B)

70. Consider the vector [1, 2, 3, 4, 5], how to build a new vector with 3 consecutive zeros interleaved between each value? (★★★) 在数组中插入等间隔的 0

hint: array[::4]

Python
# Author: Warren Weckesser

Z = np.array([1,2,3,4,5])
nz = 3
Z0 = np.zeros(len(Z) + (len(Z)-1)*(nz))
Z0[::nz+1] = Z
print(Z0)

71. Consider an array of dimension (5,5,3), how to mulitply it by an array with dimensions (5,5)? (★★★)

hint: array[:, :, None]

Python
A = np.ones((5,5,3))
B = 2*np.ones((5,5))
print(A * B[:,:,None])

72. How to swap two rows of an array? (★★★) 交换两行

hint: array[[]] = array[[]]

Python
# Author: Eelco Hoogendoorn

A = np.arange(25).reshape(5,5)
A[[0,1]] = A[[1,0]]
print(A)

73. Consider a set of 10 triplets describing 10 triangles (with shared vertices), find the set of unique line segments composing all the triangles (★★★)

hint: repeat, np.roll, np.sort, view, np.unique

Python
# Author: Nicolas P. Rougier

faces = np.random.randint(0,100,(10,3))
F = np.roll(faces.repeat(2,axis=1),-1,axis=1)
F = F.reshape(len(F)*3,2)
F = np.sort(F,axis=1)
G = F.view( dtype=[('p0',F.dtype),('p1',F.dtype)] )
G = np.unique(G)
print(G)

74. Given a sorted array C that corresponds to a bincount, how to produce an array A such that np.bincount(A) == C? (★★★)

hint: np.repeat

Python
# Author: Jaime Fernández del Río

C = np.bincount([1,1,2,3,4,4,6])
A = np.repeat(np.arange(len(C)), C)
print(A)

75. How to compute averages using a sliding window over an array? (★★★) 滚动求值,类似于rolling

hint: np.cumsum, from numpy.lib.stride_tricks import sliding_window_view (np>=1.20.0)

Python
# Author: Jaime Fernández del Río

def moving_average(a, n=3) :
    ret = np.cumsum(a, dtype=float)
    ret[n:] = ret[n:] - ret[:-n]
    return ret[n - 1:] / n
Z = np.arange(20)
print(moving_average(Z, n=3))

# Author: Jeff Luo (@Jeff1999)
# make sure your NumPy >= 1.20.0

from numpy.lib.stride_tricks import sliding_window_view

Z = np.arange(20)
print(sliding_window_view(Z, window_shape=3).mean(axis=-1))

76. Consider a one-dimensional array Z, build a two-dimensional array whose first row is (Z[0],Z[1],Z[2]) and each subsequent row is shifted by 1 (last row should be (Z[-3],Z[-2],Z[-1]) (★★★) 滚动提取值

hint: from numpy.lib import stride_tricks, from numpy.lib.stride_tricks import sliding_window_view (np>=1.20.0)

Python
# Author: Joe Kington / Erik Rigtorp
from numpy.lib import stride_tricks

def rolling(a, window):
    shape = (a.size - window + 1, window)
    strides = (a.strides[0], a.strides[0])
    return stride_tricks.as_strided(a, shape=shape, strides=strides)
Z = rolling(np.arange(10), 3)
print(Z)

# Author: Jeff Luo (@Jeff1999)

Z = np.arange(10)
print(sliding_window_view(Z, window_shape=3))

77. How to negate a boolean, or to change the sign of a float inplace? (★★★) 取相反数

hint: np.logical_not, np.negative

Python
# Author: Nathaniel J. Smith

Z = np.random.randint(0,2,100)
np.logical_not(Z, out=Z)

Z = np.random.uniform(-1.0,1.0,100)
np.negative(Z, out=Z)

78. Consider 2 sets of points P0,P1 describing lines (2d) and a point p, how to compute distance from p to each line i (P0[i],P1[i])? (★★★)

No hints provided...

Python
def distance(P0, P1, p):
    T = P1 - P0
    L = (T**2).sum(axis=1)
    U = -((P0[:,0]-p[...,0])*T[:,0] + (P0[:,1]-p[...,1])*T[:,1]) / L
    U = U.reshape(len(U),1)
    D = P0 + U*T - p
    return np.sqrt((D**2).sum(axis=1))

P0 = np.random.uniform(-10,10,(10,2))
P1 = np.random.uniform(-10,10,(10,2))
p  = np.random.uniform(-10,10,( 1,2))
print(distance(P0, P1, p))

79. Consider 2 sets of points P0,P1 describing lines (2d) and a set of points P, how to compute distance from each point j (P[j]) to each line i (P0[i],P1[i])? (★★★)

No hints provided...

Python
# Author: Italmassov Kuanysh

# based on distance function from previous question
P0 = np.random.uniform(-10, 10, (10,2))
P1 = np.random.uniform(-10,10,(10,2))
p = np.random.uniform(-10, 10, (10,2))
print(np.array([distance(P0,P1,p_i) for p_i in p]))

80. Consider an arbitrary array, write a function that extract a subpart with a fixed shape and centered on a given element (pad with a fill value when necessary) (★★★)

hint: minimum maximum

Python
# Author: Nicolas Rougier

Z = np.random.randint(0,10,(10,10))
shape = (5,5)
fill  = 0
position = (1,1)

R = np.ones(shape, dtype=Z.dtype)*fill
P  = np.array(list(position)).astype(int)
Rs = np.array(list(R.shape)).astype(int)
Zs = np.array(list(Z.shape)).astype(int)

R_start = np.zeros((len(shape),)).astype(int)
R_stop  = np.array(list(shape)).astype(int)
Z_start = (P-Rs//2)
Z_stop  = (P+Rs//2)+Rs%2

R_start = (R_start - np.minimum(Z_start,0)).tolist()
Z_start = (np.maximum(Z_start,0)).tolist()
R_stop = np.maximum(R_start, (R_stop - np.maximum(Z_stop-Zs,0))).tolist()
Z_stop = (np.minimum(Z_stop,Zs)).tolist()

r = [slice(start,stop) for start,stop in zip(R_start,R_stop)]
z = [slice(start,stop) for start,stop in zip(Z_start,Z_stop)]
R[r] = Z[z]
print(Z)
print(R)

81. Consider an array Z = [1,2,3,4,5,6,7,8,9,10,11,12,13,14], how to generate an array R = [[1,2,3,4], [2,3,4,5], [3,4,5,6], ..., [11,12,13,14]]? (★★★)

hint: stride_tricks.as_strided, from numpy.lib.stride_tricks import sliding_window_view (np>=1.20.0)

Python
# Author: Stefan van der Walt

Z = np.arange(1,15,dtype=np.uint32)
R = stride_tricks.as_strided(Z,(11,4),(4,4))
print(R)

# Author: Jeff Luo (@Jeff1999)

Z = np.arange(1, 15, dtype=np.uint32)
print(sliding_window_view(Z, window_shape=4))

82. Compute a matrix rank (★★★) 求矩阵的秩

hint: np.linalg.svd, np.linalg.matrix_rank

Python
# Author: Stefan van der Walt

Z = np.random.uniform(0,1,(10,10))
U, S, V = np.linalg.svd(Z) # Singular Value Decomposition
rank = np.sum(S > 1e-10)
print(rank)

# alternative solution:
# Author: Jeff Luo (@Jeff1999)

rank = np.linalg.matrix_rank(Z)
print(rank)

83. How to find the most frequent value in an array?求众数

hint: np.bincount, argmax

Python
Z = np.random.randint(0,10,50)
print(np.bincount(Z).argmax())

84. Extract all the contiguous 3x3 blocks from a random 10x10 matrix (★★★) 取子矩阵

hint: stride_tricks.as_strided, from numpy.lib.stride_tricks import sliding_window_view (np>=1.20.0)

Python
# Author: Chris Barker

Z = np.random.randint(0,5,(10,10))
n = 3
i = 1 + (Z.shape[0]-3)
j = 1 + (Z.shape[1]-3)
C = stride_tricks.as_strided(Z, shape=(i, j, n, n), strides=Z.strides + Z.strides)
print(C)

# Author: Jeff Luo (@Jeff1999)

Z = np.random.randint(0,5,(10,10))
print(sliding_window_view(Z, window_shape=(3, 3)))

85. Create a 2D array subclass such that Z[i,j] == Z[j,i] (★★★) 判断是否对称

hint: class method

Python
# Author: Eric O. Lebigot
# Note: only works for 2d array and value setting using indices

class Symetric(np.ndarray):
    def __setitem__(self, index, value):
        i,j = index
        super(Symetric, self).__setitem__((i,j), value)
        super(Symetric, self).__setitem__((j,i), value)

def symetric(Z):
    return np.asarray(Z + Z.T - np.diag(Z.diagonal())).view(Symetric)

S = symetric(np.random.randint(0,10,(5,5)))
S[2,3] = 42
print(S)

86. Consider a set of p matrices with shape (n,n) and a set of p vectors with shape (n,1). How to compute the sum of of the p matrix products at once? (result has shape (n,1)) (★★★)

hint: np.tensordot

Python
# Author: Stefan van der Walt

p, n = 10, 20
M = np.ones((p,n,n))
V = np.ones((p,n,1))
S = np.tensordot(M, V, axes=[[0, 2], [0, 1]])
print(S)

# It works, because:
# M is (p,n,n)
# V is (p,n,1)
# Thus, summing over the paired axes 0 and 0 (of M and V independently),
# and 2 and 1, to remain with a (n,1) vector.

87. Consider a 16x16 array, how to get the block-sum (block size is 4x4)? (★★★)

hint: np.add.reduceat, from numpy.lib.stride_tricks import sliding_window_view (np>=1.20.0)

Python
# Author: Robert Kern

Z = np.ones((16,16))
k = 4
S = np.add.reduceat(np.add.reduceat(Z, np.arange(0, Z.shape[0], k), axis=0),
                                       np.arange(0, Z.shape[1], k), axis=1)
print(S)

# alternative solution:
# Author: Sebastian Wallkötter (@FirefoxMetzger)

Z = np.ones((16,16))
k = 4

windows = np.lib.stride_tricks.sliding_window_view(Z, (k, k))
S = windows[::k, ::k, ...].sum(axis=(-2, -1))

# Author: Jeff Luo (@Jeff1999)

Z = np.ones((16, 16))
k = 4
print(sliding_window_view(Z, window_shape=(k, k))[::k, ::k].sum(axis=(-2, -1)))

88. How to implement the Game of Life using NumPy arrays? (★★★)

No hints provided...

Python
# Author: Nicolas Rougier

def iterate(Z):
    # Count neighbours
    N = (Z[0:-2,0:-2] + Z[0:-2,1:-1] + Z[0:-2,2:] +
         Z[1:-1,0:-2]                + Z[1:-1,2:] +
         Z[2:  ,0:-2] + Z[2:  ,1:-1] + Z[2:  ,2:])

    # Apply rules
    birth = (N==3) & (Z[1:-1,1:-1]==0)
    survive = ((N==2) | (N==3)) & (Z[1:-1,1:-1]==1)
    Z[...] = 0
    Z[1:-1,1:-1][birth | survive] = 1
    return Z

Z = np.random.randint(0,2,(50,50))
for i in range(100): Z = iterate(Z)
print(Z)

89. How to get the n largest values of an array (★★★) 最大的 n 个数,不用排序

hint: np.argsort | np.argpartition

参考:numpy 的 partition 和 argpartition 的定义是什么?是做什么用的?

Python
Z = np.arange(10000)
np.random.shuffle(Z)
n = 5

# Slow
print (Z[np.argsort(Z)[-n:]])

# Fast
print (Z[np.argpartition(-Z,n)[:n]])

90. Given an arbitrary number of vectors, build the cartesian product (every combinations of every item) (★★★)

hint: np.indices

Python
# Author: Stefan Van der Walt

def cartesian(arrays):
    arrays = [np.asarray(a) for a in arrays]
    shape = (len(x) for x in arrays)

    ix = np.indices(shape, dtype=int)
    ix = ix.reshape(len(arrays), -1).T

    for n, arr in enumerate(arrays):
        ix[:, n] = arrays[n][ix[:, n]]

    return ix

print (cartesian(([1, 2, 3], [4, 5], [6, 7])))

91. How to create a record array from a regular array? (★★★)

hint: np.core.records.fromarrays

Python
Z = np.array([("Hello", 2.5, 3),
              ("World", 3.6, 2)])
R = np.core.records.fromarrays(Z.T,
                               names='col1, col2, col3',
                               formats = 'S8, f8, i8')
print(R)

92. Consider a large vector Z, compute Z to the power of 3 using 3 different methods (★★★) 超大数组的三次方:快速求法

hint: np.power, *, np.einsum

Python
# Author: Ryan G.

x = np.random.rand(int(5e7))

%timeit np.power(x,3)
%timeit x*x*x
%timeit np.einsum('i,i,i->i',x,x,x)

93. Consider two arrays A and B of shape (8,3) and (2,2). How to find rows of A that contain elements of each row of B regardless of the order of the elements in B? (★★★)

hint: np.where

Python
# Author: Gabe Schwartz

A = np.random.randint(0,5,(8,3))
B = np.random.randint(0,5,(2,2))

C = (A[..., np.newaxis, np.newaxis] == B)
rows = np.where(C.any((3,1)).all(1))[0]
print(rows)

94. Considering a 10x3 matrix, extract rows with unequal values (e.g. [2,2,3]) (★★★)

No hints provided...

Python
# Author: Robert Kern

Z = np.random.randint(0,5,(10,3))
print(Z)
# solution for arrays of all dtypes (including string arrays and record arrays)
E = np.all(Z[:,1:] == Z[:,:-1], axis=1)
U = Z[~E]
print(U)
# soluiton for numerical arrays only, will work for any number of columns in Z
U = Z[Z.max(axis=1) != Z.min(axis=1),:]
print(U)

95. Convert a vector of ints into a matrix binary representation (★★★)

hint: np.unpackbits

Python
# Author: Warren Weckesser

I = np.array([0, 1, 2, 3, 15, 16, 32, 64, 128])
B = ((I.reshape(-1,1) & (2**np.arange(8))) != 0).astype(int)
print(B[:,::-1])

# Author: Daniel T. McDonald

I = np.array([0, 1, 2, 3, 15, 16, 32, 64, 128], dtype=np.uint8)
print(np.unpackbits(I[:, np.newaxis], axis=1))

96. Given a two dimensional array, how to extract unique rows? (★★★) 提取唯一的行

hint: np.ascontiguousarray | np.unique

Python
# Author: Jaime Fernández del Río

Z = np.random.randint(0,2,(6,3))
T = np.ascontiguousarray(Z).view(np.dtype((np.void, Z.dtype.itemsize * Z.shape[1])))
_, idx = np.unique(T, return_index=True)
uZ = Z[idx]
print(uZ)

# Author: Andreas Kouzelis
# NumPy >= 1.13
uZ = np.unique(Z, axis=0)
print(uZ)

97. Considering 2 vectors A & B, write the einsum equivalent of inner, outer, sum, and mul function (★★★)

hint: np.einsum

Python
# Author: Alex Riley
# Make sure to read: http://ajcr.net/Basic-guide-to-einsum/

A = np.random.uniform(0,1,10)
B = np.random.uniform(0,1,10)

np.einsum('i->', A)       # np.sum(A)
np.einsum('i,i->i', A, B) # A * B
np.einsum('i,i', A, B)    # np.inner(A, B)
np.einsum('i,j->ij', A, B)    # np.outer(A, B)

98. Considering a path described by two vectors (X,Y), how to sample it using equidistant samples (★★★)?

hint: np.cumsum, np.interp

Python
# Author: Bas Swinckels

phi = np.arange(0, 10*np.pi, 0.1)
a = 1
x = a*phi*np.cos(phi)
y = a*phi*np.sin(phi)

dr = (np.diff(x)**2 + np.diff(y)**2)**.5 # segment lengths
r = np.zeros_like(x)
r[1:] = np.cumsum(dr)                # integrate path
r_int = np.linspace(0, r.max(), 200) # regular spaced path
x_int = np.interp(r_int, r, x)       # integrate path
y_int = np.interp(r_int, r, y)

99. Given an integer n and a 2D array X, select from X the rows which can be interpreted as draws from a multinomial distribution with n degrees, i.e., the rows which only contain integers and which sum to n. (★★★)

hint: np.logical_and.reduce, np.mod

Python
# Author: Evgeni Burovski

X = np.asarray([[1.0, 0.0, 3.0, 8.0],
                [2.0, 0.0, 1.0, 1.0],
                [1.5, 2.5, 1.0, 0.0]])
n = 4
M = np.logical_and.reduce(np.mod(X, 1) == 0, axis=-1)
M &= (X.sum(axis=-1) == n)
print(X[M])

100. Compute bootstrapped 95% confidence intervals for the mean of a 1D array X (i.e., resample the elements of an array with replacement N times, compute the mean of each sample, and then compute percentiles over the means). (★★★)

hint: np.percentile

Python
# Author: Jessica B. Hamrick

X = np.random.randn(100) # random 1D array
N = 1000 # number of bootstrap samples
idx = np.random.randint(0, X.size, (N, X.size))
means = X[idx].mean(axis=1)
confint = np.percentile(means, [2.5, 97.5])
print(confint)

评论