Can Someone Guide Me With This?

[Poppins]

I'm working on learning machine-learning as it is, not as I believe it to be, but I'm running into equations like the following.

 

 

• L1 norm: , which leads to sparse solutions.
• Elastic Net: , a convex combination of L2 and L1, where  is given by 1 - l1_ratio.

 

 

 

I have no idea what any of this means. I haven't taken math since my first year in college which was a Computer Science class type overview of Excel and Excel functions. You can find these equations here. I found an absolutely amazing resource on the internet just recently which led me to this alien scripture, you can find that resource here.

 

Is this all statistics? Is it advanced algebra? What am I dealing with here? How can I learn to understand this nonsense?

[Poppins]

• L1 norm: , which leads to sparse solutions.

Elastic Net: , a convex combination of L2 and L1, where  is given by 1 - l1_ratio.

• L2 norm: ,

[Poppins]

If anyone can explain this stuff in English to me I would be extremely grateful.

[sanctus]

What exactly do you want to know? You got the defs of the two norms, a norm is in simple words is just a scalar assigned to a vector somehow related to the dimension of the vector.

 

For interpretation of the equation I would need to know what the function E and L are and their parameters too (some I guess like y_i being the dependent variable derived from the indep. vars x_i)

 

I also would guess that the last eq. in OP is the evolution of \omega to the next iteration, assuming that f(x_i)=\omega^Tx_i+b, then the updated value of \omega is just the current minus \eta times the derivative dE(w,b)/dw.

 

If I get time to read through your links I can help out more.

[Poppins]

Thanks for the help. I wish I knew a little more math. They did try to explain it a little bit in the url that I gave, but I'm a low level mathematician. I stick with arithmetic and basic algebra because that is what I know. I have been teaching myself Linear Algebra lately though.

[sanctus]

Ok read a bit throught the link, if you addtionally read what gradient descent is (http://en.wikipedia.org/wiki/Gradient_descent ) then both \eta (\gamma on the wiki link) and the update of \omega should beome clear (last eq. of OP).

 

This also helps a bit:http://en.wikipedia.org/wiki/Stochastic_gradient_descent

 

The equations in the latter link are though without the penalty function (R(\omega)), the idea is the same though with the diff that if you have a penalty function you can avoid things like parameter \omega blowing up during the iterations.