Index Network is a decentralized discovery protocol that enables personalized and autonomous information retrieval through a composable vector database and an agent-based query engine.
To achieve this, Index provides a decentralized semantic index that eliminates data fragmentation, a composable discovery protocol that allows data to be queried from multiple sources, in a user-centric manner. On top of this, it provides a real-time environment for agents that facilitates integration with algorithms and services and ensures that information acquires a fluid, social, and autonomous structure.
[!NOTE] Full documentation and additional resources are available on the Index Network documentation site →
To illustrate, here is an example of an agent message that’s possible using Index Network:
The Web App component is where you can find the code for the web application. The web app provides a user-friendly interface that allows creators to explore indexes, manage their data, and configure them.
The Indexer component hosts the code for the indexer. It acts as the backbone of the natural language operations, providing the necessary methods to consume data from Ceramic, interact with large language models. It listenes Apache Kafka to consume Ceramic Network events consumes to ChromaDB by running yarn consumer
The API repository also hosts the code for all the API operations To start, you can simply run yarn api
Full protocol documentation can be found on the Index Network documentation site →
ComposeDB on Ceramic is a composable graph database built for Web3. Here you can find the steps to deploy along with an IPFS Node: https://composedb.js.org/docs/0.6.x/guides/composedb-server
ComposeDB on Ceramic is a composable graph database built for Web3. Here you can find the steps to deploy along with an IPFS Node: https://composedb.js.org/docs/0.6.x/guides/composedb-server
We use PostgreSQL indexing feature of Ceramic Network and use KafkaConnect CDC to produce db changes to Apache Kafka and finally write data to ChromaDB via the consumer service.
This code snippet demonstrates how to interact with the protocol using the Index client. It begins by initializing the client for and authenticating with a DID session. Then, it creates an index titled "Future of publishing" and a web page titled "Post medium publishing" with a specified URL. The code adds the web page to the index and performs a natural language query using the keyword "summarize," retrieving and displaying the natural language query response.
const indexClient = new Index({
network: "testnet",
});
indexClient.authenticate(session);
// Create an index
const indexId = await indexClient.createIndex({
title: "Future of publishing",
signerPublicKey: "0x047955f0df748d..1708fc8c965",
signerFunction: "QmaD9FZJYst2Tntf9UwSd3QUeD68XpaPhKBSyy5wWLHq2m",
});
// Create a web page
const webPageId = await indexClient.createWebPage({
title: "Post medium publishing",
url: "http://www.paulgraham.com/publishing.html",
});
// Add the web page to the index
await indexClient.addIndexItem(indexId, webPageId);
// Perform a natural language query
const queryResponse = await indexClient.query({
query: "summarize",
indexes: [indexId],
});
console.log("Query response:", queryResponse);
{
"response": "This article discusses the intricacies and challenges of publishing in the modern digital era, emphasizing the importance of content quality and audience engagement. The author shares insights from personal experiences and outlines strategies for successful online publishing.",
"sources": [
{
"itemId": "kjzl6kcym7w8y7fjc89gmnkne7qpdz5ws5ryfji3i8dndjh2wxuii7z1anybovo",
"indexId": "indexIdValue"
}
]
}
First of all, thanks for thinking of contributing to this project. Before sending a Pull Request, please make sure that you're assigned the task on a GitHub issue.
And join our community to get support, provide feedback, or just to say hello on...
Index Network leverages the power of several open protocols to enhance its capabilities:
Index Network is under the MIT license. See the LICENSE for more information.