Abstract:

Machine learning systems are often inspected through 2D projections of high-dimensional representations using techniques such as t-SNE or UMAP. While these visualizations provide useful overviews of clustering and similarity, they are inherently static: they display only the existing data points and do not allow users to interactively explore a model's decision space. We present an interactive exploration system that uses a Variational Autoencoder (VAE) as a generative proxy over a model's training distribution, turning the latent space into a navigable workspace for model sensemaking. Unlike static embeddings, the proxy provides an explicit decoding path from latent coordinates to inputs, enabling interaction patterns such as continuous sampling, interpolation between anchors, and region probing. We operationalize these capabilities through a set of interactive probes that augment a familiar scatter-plot overview with generative overlays for comparing transitions between classes and examining sparsely populated regions. A within-subject formative study (N=16) comparing an interactive VAE-based method to a static t-SNE baseline shows that generative interaction substantially improves counterfactual reasoning and influences how users assess model behavior in sparse or uncertain regions, while static embeddings sometimes provide clearer boundary perception. From these findings, we derive concrete design guidelines and architectural considerations for engineering interactive AI model exploration systems using generative latent representations.

Cite (BibTeX):

@article{vanbrabantembeddings2026,
  author    = {Vanbrabant, Sebe and Thys, Jarne and Eerlings, Gilles and Luyten, Kris and Rovelo Ruiz, Gustavo and Vanacken, Davy},
  title     = {From embeddings to exploration: Engineering interactive latent space visualizations for AI model sensemaking},
  journal = {Proc. ACM Hum. Comput. Interact.},
  year      = {2026},
  number    = {EICS}
}