dl_torch

Brief

dl_torch sub-module mainly aiming at accelerating simple deeplearing algorithm construction and it's pipline construction.

APIs

bb

1. CnnCfg: a config class for simple cnn block
2. reshape: a nn.Module class for doing reshape in nn.Sequential
3. permute: a nn.Module class for doing permute in nn.Sequential
4. ScannCore: block for SCANN
5. SCANN: doing MHSA with cnn style in a 4-D tensor like a typical image input
6. PositionalEncoding: PositionalEncoding
7. PositionwiseFeedforwardLayer: PositionwiseFeedforwardLayer
8. MultiHeadAttentionLayer: MultiHeadAttentionLayer
9. FastMultiHeadAttentionLayer: warpper for FlashMHA in MultiHeadAttentionLayer style
10. OutMultiHeadAttentionLayer: MHSA for reduce dims, [b, l, c] -> [b, D]
11. EncoderLayer: transformer layer
12. OutEncoderLayer: EncoderLayer for OutMultiHeadAttentionLayer
13. Trans: Transformer
14. TransPE: transformer with positional encoding
15. OutEncoderLayerAvg: OutEncoderLayer like, but using avgpool
16. TransAvg: transformer for OutEncoderLayerAvg
17. SeparableConv2d: SeparableConv2d
18. ResBlock: ResBlock
19. ResBlockR: ResBlock like, Identity Mappings in Deep Residual Networks : exclusive gating
20. SABlock: Parallel convolution kernel with residual structure
21. SABlockR: SABlock like, exclusive gating
22. ScannBlock1d: edited from NonLocalBlock and scann_core
23. GenCnn1d: tool func, generate cnn block
24. SABlock1D: SABlock like, 3-D tensor
25. SABlock1DR: SABlock like, 3-D tensor, exclusive gating

m

1. TransCfg: config for a simple transformer
2. LayerCfg: config for a simple layer which contains cnn and transformer
3. calcu_q_len: tool func for caulcu aq_len for Out-style transformer
4. COneDLayer: 3-D tensor
5. MAlayer: 4-D tensor
6. MAvlayer: MAlayer like, with avgpool down-sampling
7. SCANlayer: MAlayer like, with SCANN
8. str2net: dict for get blocks and layers easily
9. MATTPBase: model using MA-style layer
10. COneD: COneDLayer
11. MATTP: MAlayer
12. MAvTTP: MAvlayer
13. MATTPE: MAlayer with pos_embedding
14. SCANNTTP: SCANlayer
15. MATTP_ViT: MAlayer with ViT performance

data

1. denarmalize_img: tool func for denarmalizing a image
2. SubDataSet: a high-level Dataset Loader