fdermishin
2020-11-28 18:16:29
A = np.array([0,1,1,0,1])
N = A.shape[0]
column = (A > 0).reshape((N, 1))
mask = np.ones((N, N), dtype=np.bool)
mask = np.where(column, False, np.tril(mask, -1))
mask = np.cumsum(mask, axis=0)
B = np.where(mask, 0, np.tile(A, (N, 1)))
[[0 1 1 0 1]
[0 1 1 0 1]
[0 1 1 0 1]
[0 0 0 0 1]
[0 0 0 0 1]]
解释
- 计算下三角矩阵
[[False False False False False]
[ True False False False False]
[ True True False False False]
[ True True True False False]
[ True True True True False]]
- 在A中找到一个并用False填充相应的行
[[False False False False False]
[False False False False False]
[False False False False False]
[ True True True False False]
[False False False False False]]
- 计算累积总和以将零设置为下面的所有行。这是应该清零的所有元素的掩码
[[0 0 0 0 0]
[0 0 0 0 0]
[0 0 0 0 0]
[1 1 1 0 0]
[1 1 1 0 0]]
- 重复数组AN次
[[0 1 1 0 1]
[0 1 1 0 1]
[0 1 1 0 1]
[0 1 1 0 1]
[0 1 1 0 1]]
- 掩盖其元素
[[0 1 1 0 1]
[0 1 1 0 1]
[0 1 1 0 1]
[0 0 0 0 1]
[0 0 0 0 1]]
热门帖子
热门github
1
2
3
5
6
7
8
11
13
14
15
Mamba is a new state space model architecture showing promising performance on information-dense data such as language modeling, where previous subquadratic models fall short of Transformers. It is based on the line of progress on structured state space models, with an efficient hardware-aware design and implementation in the spirit of FlashAttention.
(翻译：Mamba 是一种新的状态空间模型架构，在信息密集型数据（例如语言建模）上显示出良好的性能，而之前的二次模型在 Transformers 方面存在不足。它基于结构化状态空间模型的进展，并本着FlashAttention的精神进行高效的硬件感知设计和实现。)