Glossary
Learn more about the technologies
Sovereign AI Cloud: Locally Governed Infrastructure for AI Workloads
A sovereign AI cloud runs GPUs, data, and models inside one legal jurisdiction, under local operators and law. It keeps training data and inference traffic from falling under foreign access rules.
Chunked Prefill: Interleaving Prompt Prefills with Decode Steps
Chunked prefill splits a long prompt into smaller pieces and schedules each piece alongside ongoing decode steps. Long prompts stop stalling other users, so inter-token latency stays steady.
NVIDIA GPU Operator: Automated Kubernetes Management of GPU Software
The NVIDIA GPU Operator is a Kubernetes operator that installs and manages drivers, container toolkit, device plugin, and monitoring on GPU nodes, so clusters expose GPUs without manual node setup.
Continuous Batching: Dynamically Updating LLM Batches at Every Step
Continuous batching lets an LLM server add new requests and remove finished ones at every decoding step. GPUs stay busy, and throughput rises compared with waiting for a whole batch to finish.
Virtual GPU (vGPU): Hardware Virtualization for Shared GPU Resources
A virtual GPU (vGPU) lets a hypervisor split one physical GPU into several virtual GPUs, each assigned to a virtual machine with its own memory share, for VDI, AI, and compute.
Paged Attention: Managing KV Cache Memory with Virtual Paging
PagedAttention stores the KV cache in small fixed-size blocks mapped through a block table, like OS virtual memory. It removes fragmentation, so one GPU serves more concurrent LLM requests.
Speculative Decoding: Accelerating LLM Inference Using Draft Models
Speculative decoding speeds up LLM inference by letting a small draft model propose several tokens that the large model verifies in one pass, keeping output identical to normal decoding.
KV Cache: Storing Attention Keys and Values to Speed Up Inference
A KV cache stores the attention keys and values of tokens already processed, so an LLM computes only the newest token at each decoding step. It trades GPU memory for lower latency.
What Is Prefix-Cache-Aware Routing?
Prefix-cache-aware routing sends each LLM request to the replica that already holds the KV cache for its prompt prefix, cutting redundant prefill, first-token latency, and GPU cost. How it works and when to use it.
What Is GPU Fractioning? Sharing One GPU Across Many Jobs
GPU fractioning splits one physical GPU into isolated slices so several inference or training jobs can share one card. MIG vs time-slicing vs software fractioning, and why it matters for cost.

What is an AI Agent?
An AI agent is an autonomous system that perceives its environment, reasons through problems, and takes action to achieve goals without constant human input. Learn how they work, types, use cases, and key challenges.

What is AI Agent Memory?
AI agent memory is the ability to store, recall, and apply information from past interactions so agents make smarter decisions over time. Learn how it works, types, and why it matters for enterprise AI.

What is AI Cloud?
Discover what AI Cloud is and how it combines artificial intelligence with cloud computing to optimize operations, drive innovation, and automate processes. Learn about key AI Cloud services, use cases, and benefits.

What is Artificial General Intelligence (AGI)?
Explore what Artificial General Intelligence (AGI) is, its key characteristics, importance, challenges, and the current state of research in this comprehensive guide.

What is Zero-Shot Learning?
Discover the power of Zero-Shot Learning, an advanced machine learning technique that enables models to recognize and classify unseen objects or tasks. Learn about its key concepts, importance, challenges, and future outlook.

What is Agentic RAG?
Discover the power of Agentic RAG, an advanced AI framework that integrates retrieval-based methods with generative models for dynamic decision-making and knowledge synthesis. Learn about its components, importance, applications, and future directions.

What is AI inferencing?
Discover what AI inferencing is and how it powers real-time decision-making. Learn about its components, applications, and challenges in this comprehensive guide.

What is Kubernetes on Edge?
Discover the power of Kubernetes on Edge. Learn about the benefits, use cases, and deployment strategies for edge computing. This can enhance IoT, smart cities, and real-time data processing.
What is Kubernetes Operator?
Discover how Kubernetes Operators can streamline the deployment and management of complex applications. Learn about their benefits, components, and how they can simplify the lifecycle of large language models and other resource-intensive workloads.

What is Multimodal AI?
Learn in detail about what is multimodal ai, its use cases, enterprise applications and how to setup and implement one for the organization. CloudRaft provides AI consultations as a complimentary service, feel free to connect with an expert.
What is RAG (Retrieval Augmented Generation)?
Learn in detail about Retrieval Augmented Generation (RAG), use cases and benefits from the team of expert AI consultants at CloudRaft.
What is Content Moderation?
Discover multimodal AI, its use cases, enterprise applications, and implementation strategies. CloudRaft offers complimentary AI consultations, connect with an expert today.
Subscribe to our Newsletter
Get our posts directly in your inbox.