Investing in Pinecone | Andreessen Horowitz

With the inflection of large language models (LLMs), we are witnessing a paradigm shift in software development and the IT industry as a whole. AI is happening and a new stack is forming before our eyes. It’s like the Internet all over again, bringing into service new infrastructure components built for the new way of doing things..

There is a growing recognition that LLMs really are a new form of computer, in a sense. They can execute “programs” written in natural language (ie, prompts), perform arbitrary computing tasks (eg, write Python code or search Google), and return the results to the user in a human-readable form. This is a big problem, for two reasons:

1. A new class of apps around summary and generative content is now possible, leading to a change in consumer behavior around software consumption.
2. A new class of developers is now able to write software. Computer programming only requires proficiency in English (or another human language), not training in a traditional programming language like Python or JavaScript.

One of our top priorities at Andreessen Horowitz is to identify the companies that are building the key components of this new AI stack. We are excited to announce that we are leading a $100 million Series B pine coneto support its vision of becoming the memory layer for AI applications.

The problem: LLMs hallucinate and are stateless

A big challenge with today’s LLMs is hallucination. They give very confident answers that are factual and sometimes logically incorrect. For example, asking an LLM for Apple’s gross margin for the most recent quarter might yield a confident answer of $63 billion. The model can even back up its answer by explaining that by subtracting $25 billion in cost of goods from $95 billion in revenue, you get a gross margin of $63 billion. Of course, he is wrong on several dimensions:

• First, the revenue number is wrong because the LLM doesn’t have real-time data. You are working with outdated training data that is months or probably years old.
• Second, he picked those revenue and cost of goods numbers at random from the financial statements of another fruit company.
• Third, your calculation of gross margin is not mathematically correct.

Imagine giving this answer to the CEO of a the fortune 500 company.

All of this happens because, at the end of the day, LLMs are prediction machines trained on large amounts of third-party Internet data. Often, the information the user needs is simply not in the training set. So, the model will give the most probable and linguistically well-formed answers based on its outdated training data. We can already begin to see a potential solution to the above problem: feeding real-time, contextually relevant private enterprise data into LLMs.

The general form of this problem is that, from a systems perspective, LLMs and most other AI models are stateless in the inference step. Each time you make a call to the GPT-4 API, the output depends not more about the data and parameters you send in the payload. The model has no built-in way to incorporate contextual data or remember what you’ve asked before. Model tuning is possible, but it is expensive and relatively inflexible (ie, the model cannot respond to new data in real time). Because models don’t manage state or memory by themselves, it’s up to developers to fill the gap.

The Solution: Vector databases are the storage layer of LLMs

This is where Pinecone comes in.

Pinecone is an external database where developers can store contextual data relevant to LLM applications. Instead of sending large collections of documents back and forth with each API call, developers can store them in a Pinecone database and then pick only the few most relevant to a query determined, an approach called learning in context. It is imperative for business use cases to really flourish.

In particular, Pinecone is one vector database, which means that the data is stored in a semantically meaningful form inlays. While a technical explanation of embeddings is beyond the scope of this post, the important part to understand is that LLMs also operate with vector embeddings, so by storing data in Pinecone in this format, part of the work of AI has been pre-processed and effectively. downloaded to the database.

Unlike existing databases, which are designed for exhaustive analytical or transactional atomic workloads, the vector database (Pinecone) is designed for eventually consistent approximate neighbor search, the appropriate database paradigm for vectors of higher dimensions. They also provide APIs for developers that integrate with other key components of AI applications, such as OpenAI, Cohere, LangChain, etc. Such a well-thought-out design makes developers’ lives much easier. Simple AI tasks such as semantic search, product recommendations or feed classification can also be directly modeled as vector search problems and run on the vector database without a final model inference step . something existing databases cannot do.

Pinecone is the emerging standard for managing stateful and contextual business data in LLM applications. We believe it is an important infrastructure component, providing the storage or “memory” layer to a new stack of AI applications.

Amazing progress for Pinecone so far

Pinecone is not the only vector database, but we believe it is the leading vector database, ready now for real-world adoption, by a significant margin. Pinecone has seen 8x growth in paying customers (approximately 1,600) in just three months, including up-and-coming tech companies like Shopify, Gong, Zapier and more. It is used in a wide range of industries, including enterprise software, consumer applications, e-commerce, fintech, insurance, media and AI/ML.

We attribute this success not only to the deep understanding of the user team, the market and the technology, but also, critically, to their cloud-native product approach from the start. One of the hardest parts of building this service is providing a reliable, highly available cloud backend that meets a wide range of customer performance goals and SLAs. Through multiple iterations on the product architecture and managing many large-scale paid customers in production, this team has demonstrated the operational excellence expected of a production database.

pine cone was founded by Edo Liberty, who has long been a staunch advocate of the importance of vector databases in machine learning, including how they can enable all businesses to build use cases on top of LLMs. As an applied mathematician, he spent his career studying and implementing cutting-edge vector search algorithms. At the same time, he was a pragmatist, building core ML tools like Sagemaker on AWS and translating applied ML research into practical products that customers can use. It’s rare to see a combination of deep research and pragmatic product thinking.

Edo is joined by Bob Wiederhold, an experienced CEO and operator (formerly of Couchbase), as a partner in operations as president and COO. Pinecone also has a great team of executives and engineers with deep experience in cloud systems from places like AWS, Google and Databricks. We are impressed by the team’s deep engineering expertise, focus on developer experience, and efficient execution of GTM, and we are privileged to partner with them to build the memory layer for applications from AI.

***

The opinions expressed herein are those of the individual AH Capital Management, LLC (“a16z”) personnel cited and are not the opinions of a16z or its affiliates. Certain information contained herein has been obtained from third party sources, including portfolio companies of funds managed by a16z. Although it has been obtained from sources believed to be reliable, a16z has not independently verified this information and makes no representation as to the enduring accuracy of the information or its suitability for any particular situation. In addition, this content may include advertisements from third parties; a16z has not reviewed these advertisements and does not endorse any advertising content contained therein.

This content is provided for informational purposes only and should not be relied upon as legal, business, investment or tax advice. You should consult your own advisors regarding these matters. References to any security or digital asset are for illustrative purposes only and do not constitute an investment recommendation or an offer to provide investment advisory services. In addition, this content is not directed or intended for use by any investor or potential investor, and under no circumstances may it be relied upon in a decision to invest in any fund managed by a16z. (An offer to invest in an a16z fund will be made only by means of the private placement memorandum, subscription agreement and other relevant documentation of that fund and should be read in its entirety.) Any investment or holding company mentioned , referred to or described are not representative of all investments in vehicles managed by a16z, and no assurance can be given that investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not given permission for a16z to disclose publicly, as well as investments not announced in publicly traded digital assets). /.

The charts and graphs provided therein are for informational purposes only and should not be relied upon in making any investment decision. Past performance is not indicative of future results. The content only speaks as of the date indicated. Any projections, estimates, forecasts, objectives, perspectives and/or opinions expressed in these materials are subject to change without notice and may differ from or be contrary to the opinions expressed by others. See https://a16z.com/disclosures for important additional information.