관심있는것들

Contraction mapping we take a sequence that is convergent in that space , apply Transfomation (T) to that sequence, we get another convergence sequence that covnerges to T of x. which is limit of that sequence fixed point if apply T trasnform to x it goes back to original x Banach Fixed Point Theorem we can know that sequence is convergent to unique fixed point X star The Bellman Optimality Oper..

Transformer = uses attention elmo just concatenating hidden vectors from left and right LSTM. have 2 half blind model , suboptimal we want to single model to look left and right simultaneously gpt built for Language Model so it looks left to right = genrating language bert masked LM = replace words with mask pretrained with 2 tasks 1. feed 2 input sentence and guess is it reasonable next sentenc..

unlike other RNN, LSTM we can some how tell decoder to where you should focus (attention) positional encoding = gives network significant boost for performance top right attention = 3 connection going into it (key, value = encoding part of source sentence) , (query = encoding part of target sentence) dot product of keys and query = gives angles between these 2 vecotors , if both of them are poin..

Markov Property Once the state is known , the history may be thrown away returns discount gamma (0~1) is the present value of future rewards if close to 0 leads to mypoic (very short) evaluation if close to 1 leads to far-sighted evaluation why discount? probelm specification = immediate rewards may actually be more valuable (e.g. consider earning interest) Solution side = Mathematically conveni..

Theorem : What is Possible? unlike greedy we can explore other action (even though its Qt is not high because of Ut, uncertainty, not very picked often) how worng our estimate are going to be (Ut) intuition 1. more number we have in average the less likely we are that if we then add "u" an added amount(Xn) that this is still going to be smaller than the actual mean similarly if we pick "u" to be..

in this lecture we simplify the setting the enviromnet is assumed to have only a single state , actions no longer have long-term consequences in the enviroment actions still do impact immediate reward , not environment , other observation can be ignored because env has only 1 state exploitation = maximise performance based on current knowledge exploration = increase knowledge , exposing self to ..

Motiviation building adult brain is hard but buidling child brain and let it become adult brain might be easier what is rl can learn without examples of optimal behaviour the interaction loop , agent exectue action , agent receive observation. goal = optimise sum of rewards ,through repeated interaction , a long term goal 2 different goal 1. find solutions manufacturing robot with a specific pur..

we can apply filter and let it represent what we want , first filter can be used to filter "polite thing" and so on. we have 2 channels for each kernel size (2,3,4) -> output size is (4,5,6) -> 1 max pooling from each channel and concatenate -> concatenate whole and put it into softmax to tell positve or negative. Regularization Dropout = Create masking vector r of Bernoulli random variable with..

SQUAD evaluation F1 meause is primary , exact match: 1/0 accuracy on whether you match one of the 3 answers F1 = 2*precision*recall / (precision + recall) precision = tp/(tp+fp) , recall = tp/(tp+fn) tp = true positive == true (the model prediction is actually true) , postive ( model said it is true,positive) number of tokens that are shared between the correct answer and the prediction fp = fal..

unlike Language model we use only last output and put it into softmax to predict what kind of language is it we first extract ascii code and then we feed them to embedding matrix. def str2ascii_arr(msg): arr = [ord(c) for c in msg] return arr, len(arr) # pad sequences and sort the tensor def pad_sequences(vectorized_seqs, seq_lengths): seq_tensor = torch.zeros((len(vectorized_seqs), seq_lengths...