AI ENGINEERING | LIVING GLOSSARY | #003
VECTOR DATABASE
Where AI Stores and Searches Meaning
AIYING — Accomplishing with AI
Imagine an organization with 100,000 documents. An employee asks a question but does not know which document contains the answer.
How can an AI application find the most relevant information without depending entirely on exact keyword matches?
One important enabling technology is the vector database.
Vector databases organize numerical representations of information and support similarity-based retrieval. They are widely used in semantic search, Retrieval-Augmented Generation (RAG), recommendation systems, and knowledge assistants.
1. What Is a Vector Database?
A vector database is a data management system designed to store, organize, index, and retrieve vector representations efficiently.
A vector is an ordered collection of numbers. In AI applications, vectors are frequently produced by embedding models.
Illustrative example:
Document: Employee travel reimbursement policy
[0.21, 0.67, 0.13, 0.48, ...]
Document: Business expense guidelines
[0.20, 0.69, 0.11, 0.46, ...]
These are fictional vectors for illustration, not actual outputs from an embedding model.
When a user submits a query, the application can convert that query into a compatible vector and retrieve stored vectors that are similar.
The retrieved vectors point to the corresponding documents, products, passages, or other items.
A vector database does not automatically understand the truth of stored information. It provides retrieval capabilities based on numerical representations and associated data.
↑ Back to contents2. Why Vector Databases Matter
Traditional databases are highly effective for structured queries such as:
SELECT * FROM employees WHERE department = 'Finance';
But many AI applications require a different type of question:
This is a similarity-based retrieval problem.
Vector databases support efficient retrieval across large collections of embeddings.
| Traditional structured query | Vector similarity query |
|---|---|
| Find employee ID 125 | Find documents similar to this question |
| Filter orders by date | Find products related to this description |
| Retrieve a known record | Retrieve semantically related passages |
These approaches are complementary. Modern database and search platforms may support structured, keyword, and vector queries within the same system.
↑ Back to contents3. How Vector Databases Work
A typical vector retrieval workflow has six stages.
Step 1: Prepare information
Collect documents or other data and divide large items into suitable retrieval units.
Step 2: Generate embeddings
Use an embedding model to convert the information into numerical vectors.
Step 3: Store vectors and metadata
Store vectors together with identifiers, source references, and relevant metadata.
Step 4: Build a search index
Create an index appropriate for the application's scale and performance needs.
Step 5: Process the query
Convert a user query into a vector using a compatible embedding model.
Step 6: Retrieve results
Search the index and return relevant items, often with similarity scores and metadata.
Documents
|
v
Chunking / Preparation
|
v
Embedding Model
|
v
Vector Database
|
v
Similarity Index
User Query
|
v
Embedding Model
|
v
Similarity Search
|
v
Relevant Documents
↑ Back to contents
4. Understanding the Architecture
A production vector retrieval system may contain several components.
| Component | Purpose |
|---|---|
| Embedding model | Converts information into vectors. |
| Vector storage | Persists vectors and associated records. |
| Vector index | Supports efficient similarity retrieval. |
| Metadata | Stores attributes such as source, category, tenant, and document ID. |
| Query engine | Executes similarity queries and filters. |
| Application layer | Uses retrieved results in search, recommendations, or RAG. |
| Security controls | Enforce access permissions and data isolation. |
Not every application requires a separate specialized vector database.
Some relational databases and search engines provide vector search capabilities. The architecture should be chosen according to the application's requirements.
↑ Back to contents5. Similarity Search and Indexing
Vector databases often support nearest-neighbor search.
The objective is to find stored vectors closest to a query vector according to a selected similarity or distance measure.
Common measures
- Cosine similarity
- Dot product
- Euclidean distance
Exact versus approximate search
Exact nearest-neighbor search evaluates candidate vectors to identify the nearest results according to the chosen metric.
Approximate nearest-neighbor (ANN) methods seek faster retrieval by using specialized index structures, potentially sacrificing some recall.
Common indexing approaches include:
- HNSW — Hierarchical Navigable Small World graphs
- IVF — Inverted File indexing
- Product Quantization — Compact vector representations
These methods involve trade-offs among retrieval accuracy, memory, storage, index construction time, and latency.
6. Industry Applications
| Application | Role of vector search |
|---|---|
| Enterprise knowledge assistants | Retrieve relevant policies, manuals, and procedures. |
| RAG applications | Supply relevant external context to language models. |
| E-commerce | Find related products and support recommendations. |
| Customer support | Locate relevant knowledge-base articles. |
| Software development | Support semantic code and documentation search. |
| Research discovery | Identify conceptually related papers and passages. |
Vector search is also useful for certain image, audio, and multimodal retrieval applications when suitable embedding models are available.
↑ Back to contents7. What Must an AI Engineer Decide?
A. Do we need a vector database?
For small datasets, simple exact search may be sufficient. For larger applications, indexed retrieval may be beneficial.
B. Which database or search platform?
Options include specialized vector databases and general-purpose systems with vector search capabilities.
Examples include Pinecone, Weaviate, Milvus, Qdrant, PostgreSQL with pgvector, and Elasticsearch.
C. Which similarity metric?
The metric should suit the embedding model and the retrieval task.
D. How should documents be chunked?
Chunk size and overlap influence retrieval precision, contextual completeness, storage requirements, and cost.
E. How should metadata be used?
Metadata can support filtering by department, document type, language, date, or tenant.
F. How do we enforce security?
Retrieval must respect user authorization. A highly similar document must not be returned if the user lacks permission.
G. How do we evaluate quality?
Useful measures include Recall@K, Precision@K, retrieval latency, and end-to-end answer quality.
H. How do we manage updates?
Applications need processes for inserting, updating, deleting, and re-embedding records.
8. Practical Industry Case: Enterprise Knowledge Assistant
Consider a company with 50,000 internal documents across HR, finance, operations, and technical support.
An employee asks:
The relevant document is titled "Business Travel Accommodation Policy."
Engineering workflow
- Collect authorized company documents.
- Extract and clean the text.
- Divide documents into searchable passages.
- Generate an embedding for each passage.
- Store vectors with source and permission metadata.
- Generate an embedding for the employee's question.
- Apply authorization filters and retrieve similar passages.
- Optionally rerank the retrieved passages.
- Supply relevant passages to a language model.
- Generate an answer with source references.
- Evaluate answer quality and retrieval performance.
The vector database helps locate relevant information even when the user's wording differs from the original document.
The language model can then use the retrieved content to formulate an answer.
Important: The vector database retrieves information. It does not independently guarantee that the generated answer is correct.
↑ Back to contents9. Try It Yourself
Mini Project: Build a Small Semantic Search System
Objective: Understand how vector storage and similarity search work in a practical application.
- Prepare 10 short documents on different topics.
- Generate embeddings using a suitable model.
- Store vectors and document identifiers in a vector-capable system.
- Submit a natural-language search query.
- Generate a query embedding.
- Retrieve the three nearest document vectors.
- Display the corresponding documents.
- Compare results with keyword search.
- Test a query containing different wording but similar meaning.
- Record retrieval quality and response time.
Optional extension: Add metadata filtering so that searches return only documents belonging to an authorized category.
Reflection questions
- Did the system retrieve the expected documents?
- Were any results similar but irrelevant?
- How did keyword and vector search differ?
- What would change with one million documents?
- How would you prevent unauthorized retrieval?
10. Connected Glossary Terms
- Token: A unit processed by language-model tokenizers.
- Embedding: A numerical representation used for similarity comparisons.
- RAG: Retrieval-Augmented Generation using external information.
- Semantic Search: Information retrieval based on meaning-related representations.
- Chunking: Dividing documents into retrieval units.
- ANN: Approximate Nearest Neighbor search.
- Reranking: Reordering candidate results using additional relevance signals.
- Evals: Measuring retrieval and application performance.
Previous lessons:
Next lesson: #004 — RAG.
↑ Back to contents11. Living Update
Editorial baseline: November 2026
Vector retrieval is increasingly integrated into broader database, search, and AI application architectures.
Important developments and engineering directions to monitor include:
- Hybrid keyword and vector search
- Metadata-aware retrieval
- Multimodal vector search
- Vector compression and quantization
- Distributed indexing and scaling
- Embedding-model migration
- Retrieval evaluation and observability
- Security and tenant isolation
A vector database should not be treated as an unlimited, automatically accurate memory system.
Retrieval quality depends on the embedding model, data preparation, indexing, filtering, and application design.
Living Glossary note: Review this entry as retrieval architectures, indexing methods, and database capabilities continue to evolve.
↑ Back to contents12. Key Takeaway
Its value is not simply storing vectors. Its value lies in helping applications retrieve the right information for the right user at the right time.
Think like an AI engineer: Does our retrieval system return information that is merely similar, or information that actually helps the user accomplish the task?
↑ Back to contents13. References and Further Reading
- pgvector — PostgreSQL Vector Similarity Search
- Weaviate — Documentation
- Milvus — Documentation
- Qdrant — Documentation
- Pinecone — Documentation
- Elasticsearch — kNN Search
- Malkov and Yashunin — HNSW Research Paper
