Relevant source for background self-study

Ian Goodfellow, Yoshua Bengio and Aaron Courville:

Deep Learning MIT Press, 2016.

Motivations

Activity tables show how users map their choices or, viceversa, how available products map onto their adopters.

Essentially, a weighted binary relationship between users and films…

From tables to matrices

Let’s try again:

$U \cdot F \cdot \mathbf{r}$ ( where $\cdot$ means concatenation)

A geometric interpretation

Mathematicians see matrix B as representing a linear transformation between two linear spaces.

It is just one of the possible representations of the map—it depends on a choice for the bases for source and target (space).

What is it?

When a square matrix represents relationships between entities, such as endorsement among persons, teams defeating other teams, friends or followers on social networks, and so on, several different eigenvectors can be obtained from the original matrix, giving rise to different kinds of spectral rankings

Example: Spectral graph theory

It provides bounds for several graph features using eigenvalues of adjacency matrices.

Applications?

Well, Google PageRank algorithm is spectral graph analysis.

Early applications in Psychology, Social sciences, Bibliometrics, Economy, and Choice theory.

Example: Spectral ranking

Let $M$ be a square matrix where $m_{ij}$ represents approval or endorsement (negative values represent disapproval)

[Seely, 1949] created an index of likeability based on the ideas of diffusion: it is important to be liked by people who in turn are well-liked and so on.

my likeability index should recursively be equal to the weighted sum of of the indices of the people who like me.

Or, let’s use row vectors $\mathbf{r} = [ r_1, r_2, \dots r_n]$:

i.e., $\mathbf{r}$ is a left eigenvector of M.

It may have no solution, but matrix preprocessing can assure one exists.

Study plan

Chapter 2 of Goodfellow et al. textbook is available.

It is a refresher of notation and Linear algebra properties, no examples.

It can be read in the background of our classes.

• Phase 1: read §§ 2.1—2.7, then § 2.11.

• Phase 2: read §§ 2.8—2.10

Data Science as Linear Algebra

Q: given a user’s declared appreciation of Science fiction films, how could this be distributed to the films they have reviewed?

Data Science as Geometry

Each user experience is a vector and a point in the [hyper]space of possible film experiences.

e.g., Jill = $<0, 0, 0, 4, 4>$

non-independence example: The Jason Bourne saga

$U = \{alb, ale\}$, $F = \{Bourne-1, Bourne-2, \dots\}$

Background: determinant

Determinant: understand matrix as an area

$|A|= 0$ means that

• column vectors are not independent;

• $A$ does not admit a unique inverse, and

• it is not amenable to further processing.

a = np.array([[a11, a12], [a21, a22],  [a31, a32]])

at = zip(*a)

Matrix rank

The Rank of a matrix A is the dimension of the vector space generated by its columns. It corresponds to

Matrix inversion

$A^{-1} \cdot A =I$

where $I$ or $U$ is the unit matrix ( myI = np.eye(n)).

Eigenpairs

Matrix $A$ has a real $\lambda$ and a vector $\vec{e}$ s.t.

$\lambda$ is an eigenvalue and $\vec{e}$ an eigenvector of A.

A square matrix $A$ is called positive semidefinite iff

• $A$ is symmetric ($A=A^T$)

• $\vec{x}^T A \vec{x} \ge 0$ for any $\vec{x}$

In such case its eigenvalues are non-negative: $\lambda_i\ge 0$.