So far…

• typically data collection/integration activities yield high-dimensional data.

• Hypergeometry is counter-intuitive and invalidates distance-based analysis

• several methods seek to reduce dimensionality: SVD, PCA, LDA

• Several methods leverage linear dependencies in data

Intricated bivariate data.

In bear: chest girth vs. lenght:

• non-linear dependencies between observed values

In deer: size of fetuses:

Operational idea

n datapoints each having (up to) d dimensions: the data space

Operational idea, II

The new distances in feature space will be accounted for in the Kernel Matrix

Identity mapping: from 2-D to 2-D

$\phi(\mathbf{x}) = \mathbf{x}$

Thanks to the Kernel trick, we can now forget the original dataset and work with the kernel matrix.

Quadratic kernel

$\phi(\mathbf{x}) = (x_1^2, x_2^2, \sqrt{2}x_1 x_2)^T$

The mapping

1. Input: a dataset $D_{n x d}$ in input/data space, let’s say $\mathbb{R}^d$

2. Compute the $K_{n x n}$ inner-products matrix

3. Is K a positive semidefinite matrix?

4. Apply eigevalue decomposition

5. How many non-zero e-values? That is the new dimensionality.

6. define the Mercer mapping $\phi_M$

7. verify that in Mercer space inner products can be mapped to look-ups to the K matrix (see point 2)

Example

1. A sample from Iris, d=2.

3. eigenpairs $(\lambda_i, \mathbf{u}_i)$ are (desc. order):

$(223.95, (-.442, -.505, -.482, -.369, -.425)^T)$

$(1.29 , (.163, -.134, -.181, .813, -.512)^T)$

$(0, \mathbf{u}_3)$ discarded

$(0, \mathbf{u}_4)$ discarded

$(0, \mathbf{u}_5)$ discarded

5. compute Mercer mappings

$\phi_M(\mathbf{x}_i) = \sqrt{\Lambda} \mathbf{U}_i$

where $\mathbf{U}_i$ is the i-th row of the U matrix.

6. verify that in Mercer space inner products can be mapped to look-ups to the K matrix (see point 2)