Lec11) Convolutional Networks for NLP

we can apply filter and let it represent what we want , first filter can be used to filter "polite thing" and so on.

  

we have 2 channels for each kernel size (2,3,4)  -> output size is (4,5,6)  -> 1 max pooling from each channel and concatenate -> concatenate whole and put it into softmax to tell positve or negative.

 

Regularization 

Dropout = Create masking vector r of Bernoulli random variable with probability p of being 1.

prevents over fitting , but test time we dont do r x z we just scale final vector W with probability p.

 

Shortcut connection

first one's semantic meaning is to see deviation form doing nothing by just passing x

second one's semantic meaning is bit more complex. 

 

Batch Normalization

it is kind of doing conv block's oupt and do Z transform. tends to make scale things in same size.

 

1*1 Convolutions

pytorchZeroToAll) CNN , Advanced CNN(inception) (tistory.com)

pytorchZeroToAll) CNN , Advanced CNN(inception)

we can reduce demensionality