AI On-Prem Deployment (Video Course)

Description

• Copyright 2027
• Edition: 1st

• Online Video
• ISBN-10: 0-13-589419-0
• ISBN-13: 978-0-13-589419-4

Master onpremises GenAI: deploy; optimize; and finetune LLMs on Linux and Kubernetes to deliver private; highperformance AI; without sending your data to the cloud.

• Deploy private GenAI on your own infrastructure: Understand GenAI fundamentals and core components, choose the right LLM, and run models locally for control over data, privacy, performance, and cost.
• Build and optimize on Linux: Configure GPU drivers and container GPU support, tune CPU/GPU inference, and run inference servers (llama.cpp and vLLM) on platforms with or without GPU, with an accessible client experience using OpenWebUI.
• Production-ready workflows with Kubernetes: Add proprietary documents using RAG, prepare and run GenAI on Kubernetes (GPU workloads and non-GPU, storage, access, scaling/monitoring), and complete an end-to-end deployment that ties everything together.

Concise Overview

AI On-Prem Deployment is a hands-on video course from expert trainer Sander van Vugt that shows IT and DevOps professionals how to run generative AI on their own infrastructure, so theyre not dependent on external services beyond their own hardware. Instead of sending prompts and proprietary content to the cloud, youll learn how local deployment gives you control over data, privacy, performance, models, and predictable costs, which is especially important for regulated or critical environments, airgapped or restricted networks, and situations with limited connectivity.

The course starts with GenAI fundamentals such as core components, LLM basics, and strategies to reduce system requirements, and then moves into practical deployment on Linux covering GPU drivers, container GPU support, CPU/GPU tuning, and inference servers that run in environments without GPUs. Youll also learn how to bring private data to your LLM using approaches such as RAG, and how to run GenAI on Kubernetes with a capstone end-to-end demonstration that ties everything together, including deploying an LLM, connecting to its API, adding private data, querying from a client, and applying best practices.

Skill Level

• Beginner
• Intermediate

 Course Requirements

• No course requirements

About Pearson Video Training

Pearson publishes expert-led video tutorials covering a wide selection of technology topics designed to teach you the skills you need to succeed. These professional and personal technology videos feature world-leading author instructors published by your trusted technology brands: Addison-Wesley, Cisco Press, Pearson IT Certification, Sams, and Que. Topics include: IT Certification, Network Security, Cisco Technology, Programming, Web Development, Mobile Development, and more. Learn more about Pearson Video training at http://www.informit.com/video.



Sample Content

Table of Contents

Generative AI Fundamentals

Introduction to Generative AI

    AI solutions overview

    Generative AI use cases and features

    Core components of a GenAI solution

    Common platforms and ecosystems

    Machine Learning versus Large Language Models

    Responsible AI

    Key skills for hosting GenAI solutions

    Hardware requirements for this course

Key GenAI Components

    What is an LLM

    The inference engine

    Agents and orchestration

    API exposure and client utilities

    Lab: Exploring Huggingface

    Lab solution: Exploring Huggingface

LLMs

    Understanding LLM architecture at a high level

    Choosing the right model

    LLM families

    Using Huggingface

    Downloading models from Huggingface

    Using llama.cpp as a simple inference runtime

    Using ollama for inference

    Lab: Test-drive an LLM with llama.cpp

    Lab solution: Test-drive an LLM with llama.cpp

Reducing LLM Usage System Requirements

    GPU or CPU

    Choosing the right LLM parameters

    Picking the right inference engine

    Cold versus warm start

    Running LLMs in Podman AI Lab

    Lab: Running LLMs in Podman AI Lab

    Lab solution: Running LLMs in Podman AI Lab

Hosting Platforms Overview

    Why it makes sense to host your own AI platform

    Hosting GenAI on Linux

    Hosting GenAI on Kubernetes

    Hosting GenAI on a public cloud

    Lab: Hosting GenAI on a public cloud

    Lab solution: Hosting GenAI on a public cloud

Hosting GenAI on Linux

Tweaking Linux for AI

    Managing GPU drivers on Linux

    Installing GPU drivers on RHEL

    Adding GPU support for containers

    GPU optimization basics

    Tuning Linux for CPU-only inference

    Lab: Running llama.cpp on GPU

    Lab solution: Running llama.cpp on GPU

Running an Inference Server

    Llama.cpp versus vLLM

    Running inference servers as containers

    Requirements for using vLLM

    Running vLLM

    Basic configuration and tuning for vLLM

    Using Open WebUI

    Lab: Running vLLM

    Lab solution: Running vLLM

Adding Data to an LLM

Options for Adding Data to an LLM

    The knowledge problem

    Prompt-based injection

    Retrieval Augmented Generation (RAG)

    Adapters

    Fine-tuning

    Lab: Running Qwen with an adapter

    Lab solution: Running Qwen with an adapter

Using RAG

    Understanding the right RAG setup

    Running backend services for RAG

    Configuring Open WebUI for RAG

    Testing RAG

    Lab: Adding RAG to a private LLM

    Lab solution: Adding RAG to a private LLM

Hosting GenAI on Kubernetes and Red Hat

Preparing the Kubernetes Platform

    Kubernetes GenAI architecture overview

    Kubernetes core resources overview

    Installing a simple on-premises Kubernetes cluster

    Deploying GPU workloads with the NVIDIA plugin

    Lab: Installing Kubernetes with GPU support

    Lab solution: Installing Kubernetes with GPU support

Running GenAI on Kubernetes

    Offering persistent storage for GenAI

    Running inference on Kubernetes

    Making the inference engine accessible with Gateway API

    Scaling and monitoring

    Lab: Offering Kubernetes-based LLM inference

    Lab solution: Offering Kubernetes-based LLM inference

Red Hat AI Solutions

    Red Hat AI products overview

    Trying Red Hat AI Inference Server

Bringing it All Together

Putting it all together

    Putting it all together: Building a practical offline-capable AI-assistant for low-connectivity environments

    Preparing the platform

    Getting the right LLM

    Running the inference engine

    Connecting the Open WebUI client

    Configuring Open WebUI for RAG

    Wrap-up with best practices



Updates

Submit Errata

