관심있는것들

['emma', 'olivia', 'ava', 'isabella', 'sophia', 'charlotte', 'mia', 'amelia', 'harper', 'evelyn'] b = {} for w in words: chs = [''] + list(w) + [''] for ch1, ch2 in zip(chs, chs[1:]): bigram = (ch1, ch2) b[bigram] = b.get(bigram, 0) + 1 sorted(b.items(), key = lambda kv: -kv[1]) [(('n', ''), 6763), (('a', ''), 6640), (('a', 'n'), 5438), (('', 'a'), 4410), (('e', ''), 3983), names.txt에 이름들에 start..

a = Value(2.0, label='a') b = Value(-3.0, label='b') c = Value(10.0, label='c') e = a*b; e.label = 'e' d = e + c; d.label = 'd' f = Value(-2.0, label='f') L = d * f; L.label = 'L' L 현재 dL/dd = -2 인 상태에서 chain rule 를 통해 dL/de 를 구하려면 dL/de = (dL/dd) * (dd/de) e를 변화했을떄 미치는 L의 변화 = d를 변화했을떄 L의 변화 * e를 변화했을때 d의 변화 (local derivative) + 는 여기서 router의 역할을 하게 된다 그냥 넘어온 dL/dd 를 골고루 뿌려준다. 현재 e,c local deri..

just trying to optimize our policy even with given 2 bad condition (practical) infinty norm == just choosing max from given vectors it doesn't need to converge to optimal , we observe just few n step and evaluate if this is good policy Performance of AVI initial error == how far from optimal from my first value function our n step difference shoud be bounded by those 2 error term Perfomance of A..

actor critic = value is used to critic (update) policy deterministic policy = greedy stochastic policy = epcilon greedy, it has some randomness doesn't always generalise well = at some point it will just keep on going with learned policy even though env changed stochastic policies with function approximation agent cannot distinguish state he's in eventhough environment is MDP == partially observ..

Dynamic Programming assume a model , solve model, no need to interact with the world at all Model Free RL no model, learn value function from experience Model based RL learn a model from experience plan value functions using the learned model model RL disadvantage first learn a model , then constrcut a value function == 2 sources of approximation error (in case of model free) learn value functio..

in subsequent slide whenever there are state notations it is agent state and it is vector Classes of function approximation tabular = learn every individual space state aggregation = partition all states in to a discret set and learn like batch Linear funtion approximation tabular case = just consider feature vectors to have as many entries as therer are states and then have entry of these curre..

monte carlo control recap policy iteration 2 step = policy evalution -> policy improvment (greedy in this example) and repeat over and over until both converge model free policy iteration using action value function by considering action into account , we can have model free improvement generalised policy iteration with action value function we are not evaluating whole policy , but instead , we ..

comparing MC vs TD random walk example using td after 1 iteration only A is changing because when we go left once the value go down to zero, other case same as initial state(0 iteration , (0.5+0)/2) alpha is learning rate , it is slow when small MC has high varience requries lower learing rate, TD has lower varience than MC Batch (updating) MC and TD most likely model given this episode TD explo..

model free prediction = monte carlo algorithims no knowledge of MDP required , only samples, to learn without a model muti armed bandit right side = true action value , which is the expected reward given in action a left side = estimate at time step t , average of reward given that you have taken that action on the subsequence time steps like GD we add our error term (observed rewaerd - current ..

we want to some how see image like left above , and find the shortest path and output that path with image like right above or how long is shortest path h,v = parameters(theta) things we cal when backprop problem is how we are gonna backpropagte since "fu" is for determing some property of shortest path ,for example getting distance of shortest path, but "hv" is about extracting input graph to t..