관심있는것들

we want to some how see image like left above , and find the shortest path and output that path with image like right above or how long is shortest path h,v = parameters(theta) things we cal when backprop problem is how we are gonna backpropagte since "fu" is for determing some property of shortest path ,for example getting distance of shortest path, but "hv" is about extracting input graph to t..

Transformer = uses attention elmo just concatenating hidden vectors from left and right LSTM. have 2 half blind model , suboptimal we want to single model to look left and right simultaneously gpt built for Language Model so it looks left to right = genrating language bert masked LM = replace words with mask pretrained with 2 tasks 1. feed 2 input sentence and guess is it reasonable next sentenc..

unlike other RNN, LSTM we can some how tell decoder to where you should focus (attention) positional encoding = gives network significant boost for performance top right attention = 3 connection going into it (key, value = encoding part of source sentence) , (query = encoding part of target sentence) dot product of keys and query = gives angles between these 2 vecotors , if both of them are poin..