Week of 12.17
Good morning my friends and welcome to my blog. This week, we went to CalTech and learned about linear predictions/regressions. It was kind of difficult, but I think I kind of have an idea of what's going on.
BUT before I get started on what I learned this week, I'll explain the concept map Will and I have toiled over for the past couple of weeks. We studied matrix factorization extensively. For example, I investigated the intricacies of LU factorization (which is exclusive to factoring square matrices) and Singular Value Decomposition (SVD factorization). Each method of matrix factorization comes with its fantastic pros and cons.
LU factorization, for example, takes advantage of the Row-Echelon form of matrix simplification (which creates an upper triangular matrix U) and condenses these steps into some elementary square matrices. These elementary square matrices have the same dimensions as the original square matrix but they have a line of 1's going down diagonally from the upper left corner to the lower right corner. To represent the steps, the rows of the elementary matrix represent the row of the parent matrix that's being modified. The columns of the elementary matrix represent the row of the parent matrix that's used to modify the other row. Since you don't repeat steps with the same rows, you can condense these elementary matrices into one big elementary step matrix L, which is in the shape of a lower triangular matrix.
BUT before I get started on what I learned this week, I'll explain the concept map Will and I have toiled over for the past couple of weeks. We studied matrix factorization extensively. For example, I investigated the intricacies of LU factorization (which is exclusive to factoring square matrices) and Singular Value Decomposition (SVD factorization). Each method of matrix factorization comes with its fantastic pros and cons.
LU factorization, for example, takes advantage of the Row-Echelon form of matrix simplification (which creates an upper triangular matrix U) and condenses these steps into some elementary square matrices. These elementary square matrices have the same dimensions as the original square matrix but they have a line of 1's going down diagonally from the upper left corner to the lower right corner. To represent the steps, the rows of the elementary matrix represent the row of the parent matrix that's being modified. The columns of the elementary matrix represent the row of the parent matrix that's used to modify the other row. Since you don't repeat steps with the same rows, you can condense these elementary matrices into one big elementary step matrix L, which is in the shape of a lower triangular matrix.
Steps of LU Matrix Factorization
General shape for Upper and Lower triangular matrices
When you multiply the upper and lower triangular matrix, you can get the original parent matrix M. This process is represented by the equation LU=M. It's not surprising how this factorization process also gets its name.
I also worked on the study of Singular Value Decomposition, which is a form of matrix factorization/decomposition. There's actually two understandings of SVD, one of which is in the context of linear algebra and the other is in the context of recommender systems (which is what our group was focusing on). Because I didn't quite understand the relatively dense mathematics involved with the linear algebra aspect of SVD, I focused my research on the recommender systems applications of the decomposition process. SVD provides us with this equation:
It looks like a handful to understand, but you should treat it like a least squares/linear regression equation. The expected rating is a product of a transposed matrix qi (items) and a vector pu (users). You can find the two values and minimize the difference between that product and the true rating value by finding the minimum sum of all the users and items within the set of all data using the least squares method. (known-predicted)^2. However, if you don’t want to overfit the set (which tends to happen with SVD), you use a regularization factor to multiply the squared sum from earlier by the sum of the magnitudes of both the user and item vectors.
I found a really interesting application of SVD in modern surveilling technology in which they can take a video and take out the background of the video. You do this by first splitting the video into 2D frames, and then condensing each frame into a 1D vector. You can stack all of these vectors together to produce a matrix, and use the linear algebra method of matrix decomposition using Singular Value Decomposition to produce some matrices. One of these matrices is the foreground objects and the other are the background. As I said before, I still don't quite understand the linear algebra approach to Singular Value Decomposition, but I'm still on a process of figuring it out. In the meantime, here are some videos I made for these two topics: (I would upload YouTube links but they're jacking my audio up so you can't hear anything, so I just uploaded these files :) )
LU Factorization
SVD Factorization
In this weeks visit to CalTech, we learned about how we could use our results from the survey from a while back (remember that?) to see if our linear regression fits could make a better prediction of a feature someone might have. We originally planned to predict whether or not someone would plan out the outfits the night before based on a (-1/+1) attribute categorization from the other questions they'd answered, but it was so biased and easily predictable that Dr. Hassibi advised us against testing that. Instead, we're trying to predict whether or not we can make a good guess if someone's gonna play an instrument based on other traits/preferences they have. It's good since our current data gives us a 50/50 split.
We use different implementations of linear regression, and since we haven't completely started dipping our toes into that research, you're going to have to stay tuned for our next epic installment...after winter break!
Wish me luck on my finals!!!
Comments
Post a Comment