# Embedding

> Dense, lower-dimensional vector representations that capture latent similarity

- **Category**: Machine Learning
- **Subcategory**: Data and Feature Representation
- **Canonical URL**: https://designpattern.fyi/patterns/embedding/

---

## Description
**Intent**: Convert high-cardinality categorical data, text, images, or similar inputs into dense, lower-dimensional, continuous vectors that capture latent similarity structure.

**Context**: One-hot or sparse encodings treat every distinct value as equally different and cannot represent similarity between categories. For high-cardinality features or complex inputs like text and images, sparse representations are large and inefficient.

**Solution**: Map discrete or high-dimensional inputs into dense, lower-dimensional, continuous vectors, typically learned jointly with the rest of the model. Inputs that behave similarly end up close together in the vector space, compressing the representation while capturing similarity that raw sparse encodings can't express.



## Use Cases
- Word embeddings for NLP tasks
- Product ID embeddings in recommendation systems
- User embeddings for personalization
- Category embeddings with semantic relationships






## Trade-offs


### Advantages

- Compresses high-dimensional sparse data into dense vectors

- Captures latent similarity structure between inputs

- Enables generalization across similar categories

- More efficient than one-hot for high-cardinality features




### Considerations & Drawbacks

- Adds parameters to the model

- Needs sufficient examples per distinct value to learn meaningful vectors

- Less directly interpretable than raw features

- Requires tuning of embedding dimensionality







---
**Reference**: [Original Source](https://github.com/GoogleCloudPlatform/ml-design-patterns)