Skip to content

Review: Visualizing Data using t-SNE

Citation

  • van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9, 2579–2605.
  • JMLR (JMLR assigns no DOI)

Abstract

We present a new technique called "t-SNE" that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of Stochastic Neighbor Embedding (Hinton and Roweis, 2002) that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map. t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales. This is particularly important for high-dimensional data that lie on several different, but related, low-dimensional manifolds, such as images of objects from multiple classes seen from multiple viewpoints. For visualizing the structure of very large data sets, we show how t-SNE can use random walks on neighborhood graphs to allow the implicit structure of all of the data to influence the way in which a subset of the data is displayed. We illustrate the performance of t-SNE on a wide variety of data sets and compare it with many other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding. The visualizations produced by t-SNE are significantly better than those produced by the other techniques on almost all of the data sets.

t-SNE is the embedding Hifuku uses to lay every gene's chart on one shared map. Its design goal, preserving which points are near which rather than the exact distances between them, is the goal Hifuku's map has as well.

From SNE to t-SNE

Stochastic Neighbor Embedding (SNE) first turns the high-dimensional distances into conditional probabilities: the similarity of point \(j\) to point \(i\) is the probability that \(i\) would pick \(j\) as a neighbor under a Gaussian centered on \(i\). A low-dimensional layout defines its own neighbor probabilities, and SNE moves the map points to minimize the Kullback-Leibler divergence between the two. Because the KL divergence is asymmetric, placing true neighbors far apart is expensive while placing distant points nearby is cheap, so the cost is dominated by local structure.

t-SNE changes two things. It minimizes a single symmetric KL divergence with a simpler gradient, and, in the low-dimensional map only, it replaces the Gaussian with a heavy-tailed Student t-distribution of one degree of freedom. The heavy tail answers the crowding problem: a two-dimensional map has far too little area to hold all the moderately-distant neighbors that a high-dimensional space allows, so those points are otherwise crushed into the center and the clusters never separate. Under a heavy tail a moderate high-dimensional distance maps to a much larger low-dimensional one, which removes the spurious attraction between moderately-dissimilar points and lets clusters open up. The result reveals cluster structure at several scales, but it does not preserve global distances: the gaps between clusters are not to scale.

In Hifuku

The abstract names the case t-SNE was built for: a single map for "high-dimensional data that lie on several different, but related, low-dimensional manifolds." This is exactly the multi-chart problem. Each gene's likelihood landscape is illuminated on its own anchor chart, a low-dimensional manifold in the space of trees, and the genes are different but related, sharing taxa and a common core. The assembled map must place these several related manifolds in one frame, and t-SNE is the technique built to do it.

t-SNE was chosen while the map was being developed, where it preserved local neighborhood structure (trustworthiness) better than classical multidimensional scaling, Isomap, or locally-linear embedding on the roughly eight-dimensional marker cloud. The map's purpose is topological: neighbors on it should be neighbors in tree space, so genes that cluster share phylogenetic signal. t-SNE does not keep global distance to scale, so, heeding the caution Hillis et al. (2005) raised about any two-dimensional tree-space plot, Hifuku draws the anchor frames on the map as a graticule that shows where the projection stretches or compresses true tree distance.