# AI Inference

#### AI Inference

AI inference within the Cortensor network is at the core of the platform’s capabilities, enabling efficient and scalable AI computations through a decentralized architecture. This section delves into the mechanisms and processes that facilitate AI inference, ensuring high performance, inclusivity, and security.

**Overview**

Cortensor's AI inference leverages a distributed network of miner nodes to perform computations using advanced AI models. The system supports a diverse range of hardware, from low-end devices to high-end GPUs, ensuring broad participation and inclusivity. The primary AI models currently supported include Llama 3, available in both quantized and regular versions, allowing even lower-end devices to contribute effectively.

**AI Inference Process**

**Task Initiation**:

* Users create sessions and submit prompts through router nodes.
* Router nodes verify session parameters, including payment and model specifications, before processing the request.

**Task Allocation**:

* Router nodes dynamically allocate inference tasks to suitable miner nodes.
* Allocation algorithms consider node performance, current workload, and specific task requirements to optimize resource utilization.

**Inference Execution**:

* Miner nodes perform the assigned AI inference tasks.
* Tasks are segmented into smaller subtasks to enhance processing efficiency and balance the workload.
* Model quantization allows lower-end devices to handle inference tasks, promoting inclusivity.

**Result Submission**:

* Miner nodes submit the results securely through encrypted channels.
* Results are sent to the router nodes for initial aggregation and verification.

**Validation**:

* Validation nodes or other miner nodes verify the inference results.
* Validation methods include semantic checks, embedding comparisons, and checksum verifications.
* Users can configure the level of validation required, balancing between cost and accuracy.

**Result Delivery**:

* Validated results are delivered to users through their preferred channels.
* The router node ensures secure and efficient result delivery while maintaining user privacy.

**Security and Privacy**

**Encrypted Communication**:

* All communications within the network are encrypted to ensure data privacy and integrity.
* Router nodes manage encryption and decryption, ensuring secure interactions between clients and miner nodes.

**Validation and Verification**:

* Validation nodes verify the accuracy of AI inference results.
* Configurable validation processes allow users to specify the required level of accuracy, influencing costs and ensuring reliable outputs.

**Inclusivity through Quantization**

**Model Quantization**:

* Cortensor employs model quantization to support a diverse range of hardware, including lower-end devices.
* This inclusivity allows devices with limited computational power to perform inference tasks, enhancing the network's scalability and resource utilization.
* The focus on supporting Llama 3 models, both quantized and regular, ensures wide participation and efficient task execution across different hardware capabilities.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.cortensor.network/technical-architecture/ai-inference.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.