User:Zmoboros
From Wikipedia, the free encyclopedia
Contents |
[edit] Johnson-Lindenstrauss lemma
The Johnson-Lindenstrauss lemma asserts that a set of n points in any high dimensional Euclidean space can be mapped down into an
dimensional Euclidean space such that the distance between any two points changes by only a factor of (1 + ε) for any 0 < ε < 1.
[edit] Introduction
Johnson and Lindenstrauss {cite} proved a fundamental mathematical result: any n point set in any Euclidean space can be embedded in O(logn / ε2) dimensions without distorting the distances between any pair of points by more than a factor of (1 + ε), for any 0 < ε < 1. The original proof of Johnson and Lindenstrauss was much simplified by Frankl and Maehara {cite}, using geometric insights and refined approximation techniques.
[edit] Proof
Suppose we have a set of n d-dimensional points p1,p2,...,pn and we map them down to
dimensions, for appropriate constant C > 1. Define T(.) as the linear map, that is if
, then
. For example T could be a
matrix.
The general proof framework
All known proofs of the Johnson-Lindenstrauss lemma proceed according to the following scheme: For given n and an appropriate k, one defines a suitable probability distribution F on the set of all linear maps T:Rd − > Rk. Then one proves the following statement:
Statement: If any T:Rn − > Rk is a random linear mapping drown from the distribution F, then for every vector
we have
![Prob[(1-\epsilon)||x|| \leq ||T(x)|| \leq (1+\epsilon) ||x||] \geq 1 - \frac{1}{n^2}](../../../../math/c/7/f/c7f03d403dfbc9865a50c5385a33d58e.png)
Having established this statement for the considered distribution F, the JL result follows easily: We choose T at random according to F. Then for every i < j, using linearity of T and the above Statement with x = pi − pj, we get that T fails to satisfy
with probability at most
. Consequently, the probability that any of the (n2) pairwise distances is distorted by T by more than 1 + ε is at most
. Therefore, a random T works with probability at least
.
[edit] References
- S. Dasgupta and A. Gupta, An elementary proof of the Johnson-Lindenstrauss lemma, Tech. Rep. TR-99-06, Intl. Comput. Sci. Inst., Berkeley, CA, 1999.
- W. Johnson and J. Lindenstrauss. Extensions of Lipschitz maps into a Hilbert space. Contemporary Mathematics, 26:189--206, 1984.
[edit] Fast monte-carlo algorithms for MM
Given an
matrix A and an
matrix B, we present 2 simple and intuitive algorithms to compute an approximation P to the product AB, with provable bounds for the norm of the "error matrix"
. Both algorithms run in O(mp + mn + np) time. In both algorithms, we randomly pick s = O(1) columns of A to form an
matrix S and the corresponding rows of B to form an
matrix R. After scaling the columns of S and the rows of R, we multiply them together to obtain our approximation P . The choice of the probability distribution we use for picking the columns of A and the scaling are the crucial features which enable us to give fairly elementary proofs of the error bounds. Our rest algorithm can be implemented without storing the matrices A and B in Random Access Memory, provided we can make two passes through the matrices (stored in external memory). The second algorithm has a smaller bound on the 2-norm of the error matrix, but requires storage of A and B in RAM. We also present a fast algorithm that \describes" P as a sum of rank one matrices if B = A

