Postgres and vector storage - The rise of pgvector in RAG world | S02 EP14
Jonathan Katz, AWS Principal PM is going to join us on the hottest topic in GenAI town - vector storage in RAG (retrieval augmented generation) domain. We will dive deep into pgvector extension capabilities. John will share his perspective on pgvector improvements and how it brings GenAI workloads closer to postgres customers.
Todays show focused we discussed included the rise of vector databases, choosing a vector database engine, improvements in PG Vector over the past year, integrating AWS AI/ML services with Postgres and PG Vector, and the PG Vector roadmap.
Key highlights:
- Vector math has existed since the early 1900s but using vectors for database similarity search is relatively new.
- There has been a lot of innovation recently across different vector database options with improvements in performance, relevance, and scalability.
- PG Vector now supports indexing methods like HNSW which can be over 20x faster than previous methods and makes it easy for developers to use.
- New PG Vector features like quantization will help workloads scale to billions of vectors by reducing index size.
- Role-based security in Postgres can be combined with PG Vector and retrieval augmented generation to control which users can access which vectorized documents.
Check out the recording here:
Loading
Hosts of the show 🎤
Tony Mullen - Senior RDS Specialist Solutions Architect @ AWS
Guests 🎤
Jonathan Katz - Principal Product Manager - Technical @ AWS & PostgreSQL Core Team Member
Jay Sampath - Principal Solution Architect @ AWS
Raj Jayakrishnan - Senior Database Specialist Solutions Architect @ AWS
Links from today's episode
- Leverage pgvector and Amazon Aurora PostgreSQL for Natural Language Processing, Chatbots and Sentiment Analysis - https://aws.amazon.com/blogs/database/leverage-pgvector-and-amazon-aurora-postgresql-for-natural-language-processing-chatbots-and-sentiment-analysis/
- AWS re:Invent 2023 - Best practices for querying vector data for gen AI apps in PostgreSQL (DAT407) - https://www.youtube.com/watch?v=PhIC4JlYg7A
- Accelerate HNSW indexing and searching with pgvector on Amazon Aurora PostgreSQL-compatible edition and Amazon RDS for PostgreSQL - https://aws.amazon.com/blogs/database/accelerate-hnsw-indexing-and-searching-with-pgvector-on-amazon-aurora-postgresql-compatible-edition-and-amazon-rds-for-postgresql/
- Jonathan Katz blog - https://jkatz05.com/
- Generative AI Use Cases with Aurora PostgreSQL and pgvector - https://catalog.workshops.aws/pgvector/en-US & /github.com/aws-samples/aurora-postgresql-pgvector
- Building AI-powered search in PostgreSQL using Amazon SageMaker and pgvector - https://github.com/aws-samples/rds-postgresql-pgvector
Check out Past Shows
You can check out our past shows from out community page -
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article