User Interaction & Node Communication
In the Cortensor network, three primary node roles work together to facilitate AI inference tasks: Client Nodes, Router Nodes, and Miner Nodes. These roles interact seamlessly to ensure decentralized, efficient, and scalable AI service delivery. Below is an explanation of how communication and user-serving workflows operate within this architecture.
Node Roles Overview
Client Nodes: Client nodes represent users or applications (web2 or web3) that initiate AI tasks by interacting with the Cortensor network. These nodes can be embedded into an application to provide AI features, and they primarily communicate with the Router Nodes and smart contracts. Client nodes can bypass router nodes and interact directly with miners and the smart contract if needed, but utilizing router nodes enhances the user experience by optimizing communication and integration with the network's full features.
Router Nodes: Router nodes act as intermediaries between client nodes and miner nodes. They handle various protocols and interfaces, including:
REST API, REST API Stream, and WebSocket for web2 clients to offer smooth communication channels.
Smart contracts and decentralized storage protocols like IPFS for web3 clients, enabling decentralized interaction and result storage.
Relaying requests and responses between miners and clients in real-time, ensuring seamless communication.
Streaming responses back to client nodes, mimicking behavior like OpenAI’s LLM streaming responses for enhanced user interaction.
Authorizing client and miner nodes within the network via smart contracts.
Managing the reputation of miners based on their task execution, token speeds, and accuracy.
Router nodes also validate and forward miners' responses to clients, ensuring the network's integrity and timeliness while handling various protocols.
Miner Nodes: Miner nodes are responsible for performing AI inference tasks. When selected by the network, these nodes execute tasks such as large language model (LLM) inference or data processing. Miners are rewarded for completing tasks based on network demand and user requirements.
Flow of Operations
1. Session Creation
The process begins when a client node creates a session. A session is essentially a contract between the client and the network, specifying the AI task configuration, including which model to use, the required correctness, and the validation mechanisms. The client must deposit $COR tokens into the network, which are deducted as they utilize services. This deposit model helps predict network capacity requirements and allows for dynamic pricing in the future.
2. Task Request
Once a session is created, the client node sends a request to the smart contract. The smart contract, in turn, informs a router node, which consumes the task and assigns it to qualified miner nodes. Router nodes act as job schedulers, selecting miners based on predefined criteria, such as miner performance and reputation. Miners may be dedicated to the session, or selected randomly as ephemeral miners for short-term assignments.
3. Session Queue and Task Assignment
Router nodes maintain a session queue to manage task distribution. Miners either consume tasks from this queue on a first-come-first-serve basis, or they may be pre-assigned to specific sessions. If the client requests a higher level of correctness or redundancy, multiple miners may work on the same task to ensure accuracy. In such cases, several miners perform the same task, and their results are compared using methods such as embedding vector distance to identify discrepancies and ensure no miners are behaving dishonestly.
4. Mining and Task Execution
Miner nodes perform the actual AI inference, such as running LLM completions based on the client’s input and configuration. For complex tasks, miners may need to coordinate their efforts across multiple steps or phases, using the Proof of Useful Work (PoUW) state machine to track progress and ensure that all tasks are completed correctly and on time. If a miner fails to respond within a given period, an Oracle node requests the smart contract to assign the task to a different miner.
5. Precommit and Commit Phases
During the task execution, multiple miners may participate in the precommit and commit phases. In the precommit phase, miners submit a hash of their work to prevent dishonest behavior. During the commit phase, miners reveal their actual results, which are compared against each other to ensure consistency.
6. Result Submission and Validation
Once miners complete the task, the results are submitted back to the smart contract and stored on decentralized storage such as IPFS. These results are also streamed back to the router node, which relays them to the client node in real-time (optional, similar to OpenAI's streaming response). Validation of the task results occurs through a process similar to PoUW, where validators (or additional miners) use embedding vectors to compare results and ensure the task was performed correctly.
Mining and Network Incentives
Miners earn rewards by processing AI tasks. These rewards are based on task complexity, timeliness, and accuracy. Even during idle times, miners can perform background tasks and earn network rewards. This dual reward structure, consisting of network incentives and direct payments from user requests, ensures continuous participation from miner nodes.
Future Expansion
The framework described here can be extended to support more complex use cases, such as synthetic data generation and enhanced AI tasks. As Cortensor evolves, the Proof of Useful Work (PoUW) system will accommodate a broader range of AI-driven tasks, cementing its role as a decentralized AI job scheduler.
In summary, Cortensor's node communication and user-serving architecture is designed for scalability, flexibility, and decentralization. The seamless interaction between client nodes, router nodes, and miner nodes ensures efficient AI task execution, while smart contracts and PoUW mechanisms provide robust validation and incentives for network participants. The router nodes' ability to handle multiple communication protocols further enhances their role as a bridge between clients and miners.
Last updated