Relevant sources for background self-study

Ian Goodfellow, Yoshua Bengio and Aaron Courville:

Deep Learning MIT Press, 2016.

Jure Lescovec, Anand Rajaraman, Jeffrey D. Ullmann Mining of Massive datasets MIT Press, 2016.

Eigenpairs

Matrix $A$ has a real $\lambda$ and a vector $\vec{e}$ s.t.

$\lambda$ is an eigenvalue and $\vec{e}$ an eigenvector of A.

A square matrix $A$ is called positive semidefinite iff

• $A$ is symmetric ($A=A^T$)

• $\vec{x}^T A \vec{x} \ge 0$ for any $\vec{x}$

In such case its eigenvalues are non-negative: $\lambda_i\ge 0$.

Underlying idea, I

Eigenvectors, i.e., solutions to

describe the direction along which matrix A operates an expansion

Example: shear mapping

[[1, .27],
 [0,   1]]

The blue line is unchanged:

• an $[x, 0]^T$ eigenvector

• corresponding to $\lambda=1$

Eigen. of user-activity matrices

Eigenvectors are always orthogonal with each other: they describe alternative directions, interpretable as topic

Eigenvalues expand one’s affiliation to a specific topic.

Simple all-pairs extraction via Numpy/LA:

Caveat: e-values come normalized: $\sqrt{\lambda_1^2 + \dots \lambda_n^2} = 1$

hence multiply them by $\frac{1}{\sqrt{n}}$

Euclidean norm

$||\mathbf{x}|| = \sqrt{\mathbf{x}^T\mathbf{x}} = \sqrt{\sum_{i=1}^m x_i^2}$

Normalization

The unit or normalized vector of $\mathbf{x}$

1. keeps the directon

2. norm is set to 1.

With Maths

With Computer Science

At Google scale, $\Theta(n^\alpha)$ methods are not advised.

Ideas:

1. find the e-vectors first, with an iterated method.

2. interleave iteration with control on the expansion in value

Now, eliminate the contribution of the first eigenpair:

Repeat the iteration on $M^*$. Times are in $\Theta(dn^2)$