Hashed Feature
Bounded representation for high-cardinality categorical data using hash functions
Intent & Description
🎯 Intent
Handle categorical fields with very high or open-ended cardinality (user IDs, search queries, zip codes, free-text tags) that would blow up vocabulary-based encoding sizes and cause out-of-vocabulary issues in production.
📋 Context
When categorical features have unbounded or constantly growing vocabularies, expected cold-start categories in production, or tight memory budgets, traditional one-hot encoding becomes impractical.
💡 Solution
Apply a hash function to the category value and take the result modulo a fixed number of buckets, producing a bounded-size representation regardless of how many distinct values exist. There’s no vocabulary to store or maintain, and out-of-vocabulary values hash into a bucket like everything else. The cost is potential collisions where unrelated categories land in the same bucket.
Real-world Use Case
- User IDs or product IDs in recommendation systems
- Search queries or free-text tags in classification
- Zip codes or geographic identifiers with high cardinality
- Any categorical feature with unbounded or rapidly growing vocabulary
Source
📌 TL;DR
Hash high-cardinality categorical values into a fixed number of buckets to bound representation size and handle new categories in production
Advantages
- Bounded, low-maintenance representation regardless of cardinality
- No vocabulary to store or maintain
- Handles out-of-vocabulary values gracefully
- Memory-efficient for high-cardinality features
Disadvantages
- Non-reversible mapping (loss of interpretability)
- Collisions can inject noise and hurt accuracy if too few buckets
- Less precise than full vocabulary encoding