Session Queue
The Session Queue Module in Cortensor is a state machine-controlled task queue system designed to handle user tasks. It is responsible for managing inference requests from users, assigning ephemeral nodes to execute these tasks, and interacting with miners for AI inference processing.
While the Cognitive Module focuses on network tasks and regulates task flows at the infrastructure level, the Session Queue is dedicated to user-driven tasks, ensuring a structured and reliable execution pipeline.
Design & Functionality
The Session Queue acts as a state machine, processing AI inference tasks submitted by users through the Session Module. Miners interact with the Session Queue to consume tasks, perform AI inference, and return results.
Task Flow & State Transitions
The Session Queue follows a structured state transition model, ensuring data reliability and task integrity. The key states include:
Queued → Task is received from the Session Module and added to the queue.
Acked → Ephemeral nodes acknowledge the task and signal readiness to process.
Precommitted → Miners generate a hash of their inference results, ensuring integrity before submission.
Committed → Miners submit actual inference outputs, finalizing the process.
Completed → The task is marked as successfully processed, and results are returned to the user.
These states ensure structured task handling, prevent race conditions, and maintain data integrity throughout the AI inference workflow.
Interaction with Other Modules
The Session Queue Module acts as an intermediary between key Cortensor components:
Session Module → Pushes user tasks to the Session Queue.
Node Pool & Ephemeral Nodes → Assigns miners to execute tasks.
Router Nodes → Relay results back to users via REST API or WebSocket.
Cognitive Module → Ensures network-wide coordination but does not directly manage user tasks.
Comparison: Cognitive Module vs. Session Queue
Primary Function
Network-wide task management
User AI task execution
Task Type
Infrastructure & health-check tasks
AI inference requests
State Machine
Complex with multiple verification layers
Lighter with fewer states
Interaction With
Oracle Nodes, Miners
Session, Miners, Ephemeral Nodes
Current Design & Future Enhancements
Current Implementation
Handles real-time AI inference requests.
Manages ephemeral nodes dynamically for task allocation.
Implements structured state transitions for reliability.
Future Considerations
Task Prioritization: Enhancing scheduling to prioritize urgent AI requests.
Load Optimization: Smarter balancing across multiple miners for faster processing.
Adaptive Session Management: Allowing dynamic scaling of ephemeral nodes based on demand.
Conclusion
The Session Queue Module is a critical part of Cortensor's decentralized AI framework, ensuring efficient, structured, and scalable AI inference task management. By leveraging a state machine approach, it guarantees task integrity, secure computation, and seamless coordination with miners and ephemeral nodes.
Last updated