AI Foundations: From Zero to a Self-Trained Model experimental

AI Foundations: From Zero to a Self-Trained Model Perceptron to a self-hosted, tool-using model you built, trained and run yourself What is AI: from perceptron to LLM. The one-page mental model of the field: perceptron, linear regression, classical machine learning, deep learning, and how a large language model's lifecycle actually runs.. Place the perceptron, linear regression, classical machine learning, deep learning and large language models on one timeline and one mental model.. Explain what a 3D transformer visualization actually shows happening inside a running model.. Describe the LLM lifecycle from raw text corpus to an aligned, chat-ready model.. From perceptron to deep network, Classical ML versus deep learning, Seeing a transformer in 3D, The LLM lifecycle: data, pretraining, alignment What is AI: from perceptron to LLM Every large language model (LLM) you will run in this course, however capable it looks in a browser demo, is built out of the same handful of ideas that were already on the table in the 1950s. This lesson lays down the single mental model the rest of SYNAPSE returns to, so that later modules on hardware, serving and fine-tuning all sit on the same foundation. From perceptron to deep network In 1957 and 1958, Frank Rosenblatt described the perceptron: a single artificial neuron that takes several numeric inputs, multiplies each by a learned weight, sums them, and fires or does not fire depending on whether the sum crosses a threshold. That is the entire idea. A linear regression model is a close statistical cousin: it predicts a continuous number as a weighted sum of inputs, fitted so the weights minimise error on training data. Neither model has any notion of a "layer" stacked on another layer. Stack perceptron-like units into layers, and let each layer feed the next, and you get a neural network. By common convention, once a network has more than one hidden layer between its input and output, it earns the name deep neural network (DNN). "Deep learning" is simply the practice of training such multi-layer networks, typically on far more data and compute than a single perceptron ever needed. Classical ML versus deep learning Before deep learning became practical at scale, classical machine learning dominated: decision trees, support vector machines, and, still, linear and logistic regression. The defining trait of the classical approach is that a person hand-engineers the input features (this pixel intensity, that word count) before a comparatively simple model learns weights over them. Deep learning instead lets the network learn its own internal representations directly from raw or lightly processed data, trading human feature-engineering effort for far more data and compute. Both approaches remain useful today; a large language model is simply the deep-learning branch taken to an extreme scale. Seeing a transformer in 3D Reading equations is one way to understand a transformer, the architecture behind every modern LLM. Watching one run is another. Brendan Bycroft, a developer from New Zealand, built an interactive, fully in-browser 3D visualization of a Generative Pre-trained Transformer (GPT)-style transformer (bbycroft.net/llm) that renders the token embeddings, the self-attention heads, and the feedforward (MLP) layers as you step through a forward pass. Because a full GPT-2 or GPT-3 model is too large to bundle for a browser demo, the default walkthrough uses a tiny sorting model in the same architecture family, small enough to inspect completely while showing the exact same mechanics a full-size model uses. The LLM lifecycle: data, pretraining, alignment A model like GPT does not appear as a finished chat assistant. It goes through a lifecycle. First, a large, diverse text corpus is assembled: EleutherAI's *The Pile*, released in 2020, packaged roughly 800 gigabytes of varied text specifically to train language models, and later open efforts (Common Pile, FineWeb, Dolma among them) extended the idea at much larger scale. Second, a base model is pretrained on that corpus to predict the next token, which is how GPT-1 (2018), GPT-2 (2019), GPT-3 (2020) and GPT-4 (2023) each scaled up in turn. OpenAI has never officially disclosed GPT-4's parameter count; treat any specific figure you see quoted online as an unconfirmed rumour, not a fact. A raw pretrained model is a powerful but undirected text predictor. The step that turns it into something like ChatGPT is alignment: Ouyang et al.'s 2022 InstructGPT paper described the now-standard recipe of supervised fine-tuning on example conversations, training a reward model on human preference comparisons, and then using reinforcement learning (specifically Proximal Policy Optimization (PPO)) to steer the base model toward responses people actually prefer. Every model you will run locally in this course, from a small quantized Mistral or Qwen checkpoint to whatever you eventually fine-tune yourself in the capstone, has been through some version of this same pretrain-then-align lifecycle. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) What is a GPU: vector compute, VRAM and heat. Why a GPU is a different kind of computer, why VRAM is the number that decides what model fits, and the real risks of heat and undersized power.. Explain why a GPU's vector/matrix compute model differs from a CPU's, and why that suits neural-network arithmetic.. Compute, at least approximately, how much VRAM a given model size needs.. Identify the real thermal and power risks in a GPU-heavy build and how they are mitigated.. Vector and matrix compute, VRAM and the memory bus, CUDA cores and parallelism, Heat, power and why VRAM decides what fits What is a GPU: vector compute, VRAM and heat Before buying anything, you need to understand what a Graphics Processing Unit (GPU) actually is and which of its numbers matter. This module builds that understanding so Module 02's buying decisions rest on reasons, not marketing copy. Vector and matrix compute A CPU core is built to run one thread's varied, branching instructions as fast as possible. A GPU takes the opposite bet: thousands of small, simpler cores (NVIDIA calls them Compute Unified Device Architecture (CUDA) cores) that all execute the same instruction on many different pieces of data at once, a style called Single Instruction, Multiple Threads (SIMT). Neural-network arithmetic is almost entirely matrix multiplication: the same multiply-and-add operation repeated across enormous grids of numbers. That is exactly the workload a GPU's many parallel cores are built for, which is why training and running a neural network on a GPU can be tens to hundreds of times faster than on a CPU alone. VRAM and the memory bus A GPU's own dedicated memory, Video RAM (VRAM), must hold the model's weights and, during inference, its working state. The memory bus width (for example NVIDIA's Ray Tracing-accelerated (RTX) line, such as the RTX 4090's 384-bit bus, or the RTX 5090's wider 512-bit bus) determines how many bits move between the GPU chip and its VRAM chips each cycle: a wider bus, combined with the memory's clock speed, gives higher bandwidth, which matters because a GPU with idle compute waiting on memory transfers is wasted compute. For local large language model (LLM) work, VRAM capacity is usually the harder ceiling than bandwidth. A rough sizing rule: say: V is approximately equal to P times b times one point two. where $V$ is VRAM in gigabytes, $P$ is the parameter count in billions, and $b$ is bytes per parameter (about 2 for 16-bit floating point (FP16), about 0.5 for 4-bit quantization), with the 1.2 factor a rough allowance for overhead. A 7-billion-parameter model at FP16 needs roughly 17GB; the same model quantized to 4 bits needs roughly 4 to 5GB. CUDA cores and parallelism The RTX 4090 ships 16,384 CUDA cores on a 384-bit bus; the RTX 5090 ships 21,760 CUDA cores on a 512-bit bus. More cores mean more of that same matrix-multiply work happening in parallel, which is why generation-over-generation core-count and bus-width increases translate fairly directly into faster local inference and training, all else equal. Heat, power and why VRAM decides what fits None of this compute is free of heat. The RTX 5090 alone is rated at 575W; a card sustaining that draw for hours during a fine-tuning run needs a case with real airflow and a GPU temperature that stays comfortably below its thermal-throttle point, or its clock speed (and your training time) will suffer. NVIDIA's GPU driver stack itself has also been in transition: open-source kernel modules are now the default, recommended path on Turing-generation GPUs and newer, and outright required on the newest datacenter-class platforms, though this course treats their maturity on consumer GeForce cards specifically as still developing rather than assuming datacenter-grade confidence automatically carries over. Put together, the practical takeaway of this module is simple: before you spend money, work out how many gigabytes of VRAM your target models need (Module 02 will help you decide what those targets are), then choose hardware, cooling and a power supply sized to deliver that VRAM safely and sustainably, not just to hit a headline core count. Tensor cores and mixed-precision compute NVIDIA introduced Tensor Cores alongside its Volta architecture in 2017: specialised execution units that perform mixed-precision matrix-multiply-accumulate operations directly in hardware, distinct from the general-purpose CUDA cores described above. Most local-LLM inference and fine-tuning software automatically routes matrix multiplication through Tensor Cores when they are available, which is a large part of why a Tensor Core-equipped GPU outperforms an equivalent-CUDA-core-count GPU lacking them on exactly the matrix-multiply-heavy workload this course's models create. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Building the home lab: three paths. Mac unified memory, a discrete GPU build, or a budget rig: how to choose, and how to size the power supply, motherboard, DRAM and cooling around the choice.. Compare Apple unified memory, a discrete-GPU build and a budget/used-hardware build on their real trade-offs.. Size a power supply with a defensible headroom margin for a given CPU and GPU combination.. Explain what motherboard bus lanes, DRAM and cooling actually contribute to a home AI lab.. Three paths: Mac, discrete GPU, budget build, Sizing the power supply, Motherboard, PCIe lanes and DRAM, Cooling and case airflow Building the home lab: three paths There is no single correct home AI lab. There are three defensible paths, each with real trade-offs Module 01's hardware fundamentals let you reason about honestly. Three paths: Mac, discrete GPU, budget build Apple unified memory. Apple Silicon Macs share one large pool of memory between CPU and Graphics Processing Unit (GPU), so there is no separate, smaller Video RAM (VRAM) ceiling the way a discrete card has. That is genuinely attractive for running larger quantized models than a single consumer GPU's VRAM would allow. The trade-off is a smaller software ecosystem than CUDA and generally lower raw throughput for the largest training workloads. One honest caveat: Apple's maximum configurable unified memory has actually moved around in the last year (Mac Studio configurations at the very top end have been added and removed as Dynamic RAM (DRAM) pricing shifted), so treat any specific "current maximum" figure as something to check at purchase time, not a fact to memorise from this lesson. A discrete NVIDIA GPU build. This is the path with the deepest software ecosystem (NVIDIA's Compute Unified Device Architecture (CUDA), and everything built on it) and, card-for-card, usually the best raw throughput per dollar for both training and inference. The cost is a hard VRAM ceiling per card (Module 01 showed you how to size against it) and the power, cooling and driver considerations Module 03 covers. A budget or second-hand build. Older or used GPUs, or a machine with more modest specifications throughout, trade top-end throughput and headroom for a much lower entry cost. This path leans harder on quantized models (Module 04) and realistic expectations about what fits, but it is a completely legitimate way to start. This course's approach to buying advice, throughout: treat any specific configuration or price as an illustrative, dated snapshot, useful for calibration, not a permanent recommendation. Hardware and prices move faster than a course can be rewritten. Sizing the power supply Once you know your CPU and GPU (or GPUs), size the power supply (PSU) with a simple rule: say: W subscript p s u equals the quantity T subscript c p u plus the sum of T subscript g p u, times one point two five. Sum the manufacturer's stated thermal design power (TDP) for the CPU and every GPU, then add roughly 25 percent headroom. That headroom absorbs transient power spikes under load, keeps you comfortably inside the PSU's efficiency sweet spot, and avoids the random shutdowns that an undersized supply produces under sustained training. A dual-high-end-GPU rig commonly lands in the 1500 to 1600W range once you run this arithmetic. Motherboard, PCIe lanes and DRAM A motherboard's PCIe lane count mostly matters once you add a second or third GPU, or move large datasets on and off the card repeatedly; a single GPU doing local inference rarely saturates even a modest lane allocation. Confirm the chipset actually supports your chosen CPU and the number of GPUs you intend to install, and that there is enough physical slot spacing for the cards' thickness. System DRAM is a separate resource from GPU VRAM: it holds the operating system, host processes, and staging data, not what the model's matrix multiplications actually run on, so more DRAM does not substitute for insufficient VRAM. Cooling and case airflow Cooling exists to do one thing: keep every component below the point where it throttles its own clock speed to protect itself. A case with genuinely obstructed airflow, or cards crammed too close together, will throttle under sustained load exactly when you need full performance, during a long fine-tuning run. This is not cosmetic; it is the difference between a rig that performs at its rated speed and one that quietly underperforms every benchmark you compare it against. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Choosing and installing the OS: Proxmox or Pop!_OS. A type-1 hypervisor with GPU passthrough versus a bare-metal Linux desktop, and the current state of NVIDIA drivers on Linux.. Configure GPU passthrough on Proxmox VE and contrast the result with a bare-metal desktop distribution such as Pop!_OS.. Explain GPU passthrough and its practical limits on consumer graphics cards.. Describe the current state of NVIDIA's open-source kernel modules and why that matters for a home AI lab.. Bare metal versus a hypervisor, Proxmox VE and GPU passthrough, Pop!_OS as a bare-metal AI workstation, NVIDIA drivers and open kernel modules Choosing and installing the OS: Proxmox or Pop!_OS Hardware chosen, the next decision is what runs on it. This module compares a type-1 hypervisor against a bare-metal desktop distribution, and looks honestly at where NVIDIA's Linux driver story stands today. Bare metal versus a hypervisor Running your AI workload directly on the host operating system (OS) is the simplest path: no virtualization layer stands between your model and the Graphics Processing Unit (GPU). Running it inside a hypervisor's virtual machine adds a layer of isolation and flexibility, at some cost in setup complexity. Neither is universally correct; the choice follows from how many separate things you want to run on one machine. Proxmox VE and GPU passthrough Proxmox VE, where VE stands for Virtual Environment (VE), is a Debian-based, open-source type-1 hypervisor combining the Kernel-based Virtual Machine hypervisor (KVM) for full virtual machines and Linux Containers (LXC) for lightweight containers. Its appeal for a home AI lab is running several isolated experiments or services on one physical box, and, through GPU passthrough, handing one virtual machine near-native access to a physical GPU. The important limitation to know before planning around it: consumer GeForce cards do not support splitting a single GPU across multiple simultaneous virtual machines (vGPU); passthrough on a GeForce card is strictly 1:1, one card to one virtual machine (VM) at a time. Proxmox also gives you snapshots, so you can checkpoint a working configuration before a risky driver or software change and roll back if it breaks. Pop!_OS as a bare-metal AI workstation Pop!_OS, System76's Ubuntu-based desktop Linux distribution, is a common choice when one machine is dedicated to one AI workload and the isolation a hypervisor offers is not needed. Pop!_OS ships a dedicated NVIDIA ISO with drivers preloaded, and its newer default desktop, marketed simply as COSMIC (COSMIC), is a Rust-built environment that reached stable status as the distribution's default. For a single-GPU, single-purpose workstation, bare metal avoids virtualization overhead entirely, at the cost of losing the snapshot-and-isolate workflow Proxmox offers. NVIDIA drivers and open kernel modules NVIDIA has been moving its Linux driver stack toward open-source kernel modules, which are now the default, recommended option on Turing-generation GPUs (NVIDIA's Ray Tracing-accelerated (RTX) 20-series) and every generation since, and are mandatory rather than optional on the newest datacenter-class platforms. This is a genuine, verifiable improvement in transparency and long-term maintainability over the old closed-source kernel module. Being precise matters here: this course found no clear 2026 statement confirming the open modules are production-graded specifically for consumer GeForce cards in the same sense NVIDIA states for datacenter hardware, so treat the consumer-card story as "default and actively improving" rather than "identical to the datacenter guarantee," and check current release notes before committing a build around it. Whichever path you choose, confirm your specific GPU and motherboard combination is actually supported by your chosen driver and, if relevant, passthrough approach, before buying. Consumer-card virtualization support varies by chipset and Basic Input/Output System (BIOS) firmware in ways a course cannot fully anticipate for your exact hardware. IOMMU and virtualization extensions GPU passthrough additionally requires the motherboard and processor to support an input-output memory management unit (IOMMU), a hardware feature that must be enabled explicitly in the BIOS before Proxmox VE can isolate a physical GPU's memory-mapped I/O for exclusive assignment to one virtual machine. A system without IOMMU support, or with it left disabled in firmware, cannot perform GPU passthrough at all, regardless of how correctly the hypervisor itself is configured. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Your first local LLM: Ollama and Open WebUI. Installing Ollama, understanding GGUF and quantization, and putting a self-hosted, Claude-like chat interface in front of your own model.. Install and run a quantized open-weight model locally with Ollama.. Explain what GGUF and quantization actually trade off.. Stand up Open WebUI as a self-hosted chat interface for a local model.. Ollama and the local model runner, llama.cpp, GGUF and quantization, Open WebUI: a self-hosted chat interface, Choosing your first model Your first local LLM: Ollama and Open WebUI With hardware and operating system (OS) in place, this module gets a real model talking back to you, entirely on your own machine. Ollama and the local model runner Ollama is a local large language model (LLM) runner and command-line tool that downloads, manages and serves open-weight models. Under the hood it is built on llama.cpp, and it documents genuinely modest minimum requirements: roughly 8GB of RAM and about 10GB of free disk space for a small model, on a supported operating system, with a Graphics Processing Unit (GPU) helpful but optional. Pulling and running a model is a single command; Ollama handles fetching the right weights file and starting a local server for you. llama.cpp, GGUF and quantization llama.cpp is the C/C++ inference engine underneath Ollama (and much of the wider local-LLM ecosystem), created by Georgi Gerganov. It reads models in the GPT-Generated Unified Format (GGUF) (GPT here abbreviates Generative Pre-trained Transformer (GPT)), a single-file format that packages a model's weights, often in a quantized form. Quantization reduces the numerical precision used to store each weight (from, say, 16-bit floating point down to 4 bits), trading some accuracy for a large reduction in file size and the memory needed to run the model. Unless you specify otherwise, Ollama commonly pulls a model at a 4-bit K-quant, often labelled Q4_K_M, which is usually a sensible default balance of quality against footprint. Open WebUI: a self-hosted chat interface Talking to a model through raw API calls works, but a chat interface is more natural for everyday use. Open WebUI is a self-hosted, ChatGPT-like web frontend that connects to Ollama, or to any OpenAI-compatible backend, adding conversation history, system prompts, and the ability to switch between locally installed models, all through a familiar browser user interface (UI) running entirely on your own hardware. The property worth naming explicitly: nothing in this stack sends your conversation anywhere unless you deliberately configure it to call an external API. That is a genuinely different privacy posture from a cloud chat product, and it is the foundation the rest of this course's automation, voice and capstone modules build on. Choosing your first model Pick a first model that comfortably fits your Video RAM (VRAM) headroom from Module 01, not the largest one you can find instructions for online. If an 8GB machine refuses to run a 70-billion-parameter model, the honest diagnosis is not a broken installation: even heavily quantized, a model that size needs meaningfully more memory than an 8GB minimum-spec machine provides. Start smaller, confirm the whole stack (Ollama, then Open WebUI) works end to end, and grow from there as Module 06 introduces the wider open-weight landscape you can choose from. Context window limits Every model, once loaded, operates within a fixed context window: the maximum number of tokens, roughly three-quarters of a word each on average for English text, it can consider at once across the whole conversation history and its reply. Ollama and llama.cpp let you configure a smaller working context window than a model's maximum trained context to save memory, but a conversation that grows past the configured limit will have its earliest turns silently dropped from the model's view unless the client manages that truncation explicitly. Partial GPU offloading When a model is too large to fit entirely in VRAM, llama.cpp, and therefore Ollama, can split it layer by layer, keeping as many layers as fit on the GPU and running the remainder on the CPU. This partial offloading still beats running the whole model on the CPU alone, since the GPU-resident layers still benefit from parallel matrix-multiply hardware, but overall throughput drops sharply compared with a model that fits entirely in VRAM, because every token-generation step now has to cross the comparatively slow link between CPU and GPU memory for the CPU-resident layers. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Voice: recognition, synthesis and multimodal models. Whisper and whisper.cpp for speech recognition, the open text-to-speech landscape, and what it means for a model to become multimodal.. Run offline speech recognition locally with whisper.cpp.. Choose a current, license-appropriate open text-to-speech option.. Explain what a multimodal model adds and where the current openness and licensing lines sit.. Speech recognition: Whisper and whisper.cpp, Text-to-speech: the open landscape, Multimodal models: beyond text, Putting a voice on your local assistant Voice: recognition, synthesis and multimodal models Text is only one modality. This module adds ears and a voice to the stack you built in Module 04, and introduces the wider idea of a multimodal model. Speech recognition: Whisper and whisper.cpp OpenAI's Whisper is a widely used speech-recognition model; whisper.cpp is a C/C++ port of it that runs fully offline, cross-platform, without needing Python or even a Graphics Processing Unit (GPU). Running speech recognition locally matters for the same reason running your large language model (LLM) locally does: your audio never leaves the machine, and the whole pipeline works with no internet connection and no API key. Text-to-speech: the open landscape Open text-to-speech has genuinely reshuffled in the last two years and is worth checking carefully rather than assuming an older tutorial is still accurate. Coqui Inc., the company originally behind the popular Coqui text-to-speech (TTS) project, shut down in January 2024; its codebase is kept alive today through a community-maintained fork rather than an actively supported company. Its well-known XTTS-v2 (XTTS) voice-cloning model carries a non-commercial license (CPML), and with the original company gone there is currently no vendor selling a commercial license for it. Newer entrants worth knowing include the Piper family of voices and models such as Kokoro. Some newer voice-cloning models add a watermark to their output specifically to make AI-generated speech detectable after the fact, a deliberate mitigation against misuse of cloned voices, and a licensing detail worth checking before you use any model to clone a specific real person's voice, commercially or otherwise. Multimodal models: beyond text A multimodal model extends a text-only LLM with the ability to take in and, in some cases, produce more than one type of data: images, audio, or video alongside text. Rather than expecting one enormous model to natively handle every modality equally well, the practical pattern this course teaches is composition: connect a purpose-built speech-recognition model, a purpose-built text-to-speech model, and your core LLM, each doing the one job it is actually specialised for. Putting a voice on your local assistant Put together, a fully local voice assistant pipeline looks like this: Every stage in that diagram can run on the hardware you built in Modules 01 through 03, with no cloud dependency. The reasons to prefer this over a cloud speech API are concrete rather than abstract: no per-call cost, no audio data leaving your premises, and it keeps working with no network connection at all. It is not automatically more accurate than every commercial cloud service in every case, and you should not claim it is; what it reliably offers is control, privacy and cost, which is the trade a home lab is built to make. Real-time factor and latency A local voice pipeline is only as responsive as its slowest stage, commonly measured by real-time factor: a speech-recognition model with a real-time factor of 0.5 processes ten seconds of audio in five seconds, faster than real time, while a factor above 1.0 means transcription falls further behind the live audio the longer it runs. whisper.cpp's smaller model sizes trade transcription accuracy for a real-time factor comfortably below 1.0 on modest hardware, which matters far more for a live voice assistant than for offline batch transcription of a pre-recorded file. Wake-word detection A fully local voice assistant commonly adds a lightweight wake-word detector: a small model trained to recognise one specific phrase, such as "hey computer," continuously and cheaply, running ahead of the much heavier speech-recognition step so that whisper.cpp only has to transcribe audio once a wake word has actually been heard. This two-stage design keeps the expensive transcription model idle, and therefore power- and compute-frugal, for the overwhelming majority of the time a voice assistant is simply listening for its own name. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) The open model landscape: Switzerland, Mistral, Qwen and Hugging Face. A worked case study on Switzerland's fully open Apertus model and Mistral AI, Alibaba's Qwen family for comparison, a brief survey of the wider open-weight landscape, and how to actually evaluate a model on Hugging Face.. Explain what makes Switzerland's Apertus unusually open compared with most open-weight model releases.. Compare Mistral AI's and Qwen's current open-weight lineups and licensing.. Evaluate an unfamiliar model on Hugging Face using durable, transferable criteria rather than a memorised leaderboard position.. Switzerland's Apertus: a fully open model, Mistral AI's open-weight lineup, Qwen and the wider open-weight landscape, Reading a model card on Hugging Face The open model landscape: Switzerland, Mistral, Qwen and Hugging Face Which model should you actually run? This module treats that question the right way: not as a fixed answer to memorise, but as a skill in reading a model's provenance, license and community support, illustrated by a genuine case study. Switzerland's Apertus: a fully open model In September 2025, a Swiss consortium of the Swiss Federal Institute of Technology Zurich, abbreviated ETH (ETH), the Ecole Polytechnique Federale de Lausanne (EPFL) and the Swiss National Supercomputing Centre (CSCS) released Apertus, in 8-billion and 70-billion-parameter sizes, under the Apache 2.0 license. What makes Apertus genuinely unusual, even among "open-weight" releases, is how much of the pipeline it actually publishes: not just the final weights, as most open-weight releases do, but the training data, the training code and intermediate checkpoints too. That is a meaningfully more open standard than most of the field, and worth treating as a benchmark for what "open" can actually mean, rather than assuming every model called open-weight has released the same things. Mistral AI's open-weight lineup Mistral AI, a French company, has become one of the most consistently active open-weight labs, releasing a lineup that spans large frontier-scale models down to small edge-sized ones, with its flagship line having moved to the Apache 2.0 license. As with any fast-moving lab, the specific current flagship model name is less durable knowledge than the pattern: check Mistral's own release announcements for what is current when you actually pick a model, rather than trusting a name fixed in this lesson. Qwen and the wider open-weight landscape Qwen, Alibaba's open-weight model family, has followed a similar pattern of frequent generational releases, consistently under the Apache 2.0 license, spanning a wide range of sizes including mixture-of-experts variants. Beyond Switzerland, Mistral and Qwen, the open-weight landscape includes several other consistently active families worth knowing by name: Meta's Llama family, Google's Gemma family, and DeepSeek's models. Each has its own licensing terms and release cadence; the durable lesson is to check the current state directly rather than assume last year's comparison still holds. Reading a model card on Hugging Face Hugging Face is the de facto hub where almost all of these open-weight models are published, and reading a model card well is the actual transferable skill this module teaches. Check, in order: the license field, since it determines what you may legally do with the weights; whether an active quantizer account (community uploaders such as bartowski, mradermacher or unsloth, who have effectively succeeded the once-dominant TheBloke) has already published GPT-Generated Unified Format (GGUF) versions (GPT here abbreviates Generative Pre-trained Transformer (GPT)), which tells you both that the model is popular enough to be worth quantizing and that a ready-to-run local version likely exists; and the model's stated training data and provenance, where available. One thing worth knowing about benchmarks specifically: Hugging Face's original "Open large language model (LLM) Leaderboard" has been retired and archived as a frozen historical snapshot, not a live ranking; the ecosystem has moved to a wider set of community leaderboards instead. If you find two GGUF repositories for what looks like the same base model with very different file sizes, the near-certain explanation is that they are different quantization levels of the same underlying weights, not two different models: smaller files trade numerical precision for a lighter footprint. The single durable takeaway of this module: "which model is best" changes every few months in this field. The skill that does not go stale is knowing how to open a model card and judge it yourself. Licenses beyond Apache 2.0 Not every open-weight model uses the permissive Apache 2.0 license this module's featured families favour: Meta's Llama family ships under its own custom community license with use restrictions above a stated monthly active user threshold, and some research-oriented releases use non-commercial licenses that forbid production or revenue-generating use entirely. Reading the actual license text on a model's Hugging Face page, rather than assuming every open-weight model can be used identically, is exactly the transferable skill this module is teaching, since the practical difference between Apache 2.0 and a restrictive research license can be the difference between a model you can ship and one you cannot. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Automation: the OpenAI-compatible API, curl, JS and Go. Calling a local model as a plain HTTP and JSON contract from curl, JavaScript and Go, building a minimal single-page chat app, and gating structured output for quality.. Explain why an OpenAI-compatible endpoint lets the same client code call a local model or a cloud model, then call it from curl, JavaScript and Go.. Build a minimal single-page chat application against that endpoint.. Design a quality gate that validates structured JSON output before it is trusted or forwarded.. Calling a local model like any OpenAI-compatible API, From curl to JavaScript and Go, Building a minimal single-page chat app, Structured JSON output and quality gates Automation: the OpenAI-compatible API, curl, JS and Go This module uses JavaScript (JS) and Go, alongside curl, to turn your local model from something you chat with into something other programs can call. The key fact underneath everything here: both Ollama and llama.cpp expose an OpenAI-compatible HTTP API, so the same client code written against OpenAI's cloud service runs against your own local model, usually by changing nothing more than the base URL. Calling a local model like any OpenAI-compatible API Underneath any software development kit (SDK), a call to /v1/chat/completions is nothing more than an HTTP POST (POST) request carrying a JavaScript Object Notation (JSON) body (the model name, the conversation so far as a list of messages, and generation parameters) and receiving a JSON response containing the model's reply. Once you can see that plainly, the SDK is a convenience, not a requirement. From curl to JavaScript and Go Writing the identical call first in curl, then in JavaScript (fetch), then in Go (net/http) is deliberate: it proves the contract is a plain HTTP and JSON interface, not something tied to one language or one company's SDK. JavaScript suits a browser-based single-page app; Go suits a small, single-binary automation tool or backend service. Both send the same JSON body to the same endpoint and parse the same JSON reply. Building a minimal single-page chat app A working single-page chat app needs surprisingly little: a text input for the user's message, a fetch call to your local endpoint carrying the growing conversation history, and code that appends the returned message to the page. Everything from Open WebUI's richer feature set (Module 04) is an elaboration of exactly this loop, not a different mechanism. Structured JSON output and quality gates Free-form prose is easy for a human to read and hard for a program to parse reliably. Asking the model to return structured JSON against an explicit schema lets downstream code parse a fixed, predictable shape instead of pattern-matching unpredictable natural language. That alone is not enough: a quality gate, this course's term for a measurable, checkable pass/fail condition applied to output before it is trusted or forwarded, is what actually makes structured output safe to automate around. A serviceable quality gate validates the JSON against its expected schema, checks required fields are present, and sanity-checks values against expected ranges, retrying or rejecting the response on failure rather than forwarding whatever came back. The module's closing point is a discipline, not a trick: never let an automation pipeline trust a model's output silently. A malformed JSON reply that slips through unvalidated into a downstream system is not a rare edge case; it is the default failure mode of any automation built without a quality gate, and it is exactly the failure Module 09's capstone will ask you to guard against in a long-running pipeline. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Giving your model search and tools. Why a local model needs live retrieval, the self-hosted metasearch pattern, retrieval-augmented generation, and the tool-calling pattern behind the Model Context Protocol.. Explain why a local model needs a connected search or retrieval layer.. Describe how a self-hosted metasearch engine and retrieval-augmented generation reduce stale or invented answers.. Describe the general shape of a standard tool-calling interface for an LLM application.. Why a local model needs search, Self-hosted metasearch: the pattern, Retrieval-augmented generation, Tool use and the MCP pattern Giving your model search and tools A model's weights are frozen at the moment training ended. This module gives your local assistant a way to reach outward, past that frozen snapshot, for current information and real actions. Why a local model needs search Every model you have run so far in this course knows only what was in its training data, up to whatever date that data was collected. It cannot see today's news, your private files, or anything that happened after its training cutoff, no matter how confidently it answers. Left unaddressed, this is the root cause of the most common failure mode of a local assistant: a confident, plausible, wrong answer about anything current, sometimes called hallucination when the model fills the gap with an invented but fluent-sounding response. Self-hosted metasearch: the pattern The general pattern that fixes this is straightforward: give the model, or the application wrapped around it, a way to query a search engine and read the results. A self-hosted metasearch engine aggregates results from multiple search backends behind one interface you run yourself, without the querying activity being tracked by a third party; SearXNG is a well-known, actively documented open-source project in this space and is a reasonable starting point to read about, though the exact tool you eventually adopt matters less than understanding the pattern itself. This lesson is honest about one limitation: if you have a specific search tool in mind from something you read or half-remembered, verify its actual name and current documentation yourself before building around it, rather than trusting a course to have guessed correctly at a name it could not independently confirm. Retrieval-augmented generation Retrieval-augmented generation (RAG) takes the same idea one step further: instead of, or alongside, a live web search, fetch relevant text from a document collection you control (your notes, a knowledge base, a set of PDFs) at query time, and insert it directly into the model's context before it generates a reply. A minimal working RAG pipeline needs three things: a way to fetch or retrieve relevant snippets, a way to insert that text into the prompt, and the underlying local model itself to reason over it. RAG does not require a paid vector-database subscription; a home-lab implementation can start with something far simpler and grow only if it needs to. Tool use and the MCP pattern Beyond search, a model can be given access to arbitrary tools: run a calculation, read a file, call another program. The Model Context Protocol (MCP) is a standard way for a large language model (LLM) application to discover and call such external tools and data sources through one common interface, rather than writing a bespoke, one-off integration for every tool a model might need. Where Module 07 taught you to call a model as an API, this module teaches the model itself to reach outward, deciding when to call a search or a tool as part of producing its answer. The same discipline from Module 07 still applies here without exception: whatever a search or tool call returns should still be validated and sanity-checked before your application trusts or forwards it. A tool result is not automatically more reliable than a raw model output just because it came from outside the model; it is simply a different kind of untrusted input that deserves the same quality gate. Prompt injection through retrieved content Once a model can read content it did not generate, whether a web search result or a retrieved document, that content becomes an untrusted input capable of prompt injection: text crafted to look like an instruction the model should follow rather than data it should merely reason about, for example a web page containing a hidden instruction telling the model to ignore its system prompt and take some other action. The same quality-gate discipline this module and Module 07 apply to a model's output must also be applied, in the other direction, to anything retrieved and inserted into a prompt before it reaches the model, since an unvalidated search result or document chunk is exactly the kind of untrusted input a tool-using pipeline should never trust by default. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/) Capstone: your own expert model, a chatbot, a stream processor. Specialise a base open-weight model into a stated, quality-gated expert model, wrap it in a chatbot, and build a live stream processor, drawing on every earlier module.. Explain the trade-off between full fine-tuning and parameter-efficient fine-tuning, then fine-tune or adapt a base open-weight model toward a stated, narrow task with a measurable quality gate.. Wrap a local model in a working chatbot interface.. Build a stream processor that applies the model continuously to an incoming feed.. Choosing what to fine-tune and why, From base model to expert model, Wrapping it in a chatbot, A live stream processor: continuous automation Capstone: your own expert model, a chatbot, a stream processor Every earlier module in SYNAPSE was building toward this one. The capstone asks you to specialise a model, wrap it usefully, and prove both actually work, with the same honesty and quality-gate discipline the whole course has practised. Choosing what to fine-tune and why "Train your own model" and "fine-tune an existing open-weight model" are not the same undertaking. Pretraining a model from randomly initialised weights needs data and compute far beyond any home lab this course describes; that is not false modesty, it is the actual scale gap between a frontier lab's training run and a consumer Graphics Processing Unit (GPU). The realistic, valuable path is fine-tuning: continuing training of an existing open-weight base model (Module 06) on a smaller, focused dataset so it specialises toward one particular task or domain. Choose a narrow, well-defined task you can actually measure, not a vague ambition to make the model "generally smarter." From base model to expert model Full fine-tuning updates every weight in the model, which needs Video RAM (VRAM) comparable to training the model from scratch. Parameter-efficient fine-tuning (PEFT), methods such as LoRA under the Hugging Face PEFT umbrella, instead freeze the base model and train only a small set of added parameters, making real specialisation possible on far less VRAM than full fine-tuning would need, which is usually the realistic choice for the hardware built in Module 02. Once trained, an Ollama Modelfile lets you package a custom system prompt, generation parameters, or your fine-tuned weights layer as a single runnable local model, exactly like any base model from Module 04. The non-negotiable part of this section is the quality gate: your capstone specialisation is only worth keeping if you can show, on a stated, measurable task, that it outperforms the unmodified base model. Without that comparison, added complexity has no justification, and an honest capstone write-up says exactly what improved, on what task, under what conditions, not that the result is now a general-purpose frontier model. Wrapping it in a chatbot At minimum, "a chatbot" for this capstone means the same loop Module 07 built: a local model behind an OpenAI-compatible endpoint, wrapped in an interface that keeps conversation history and lets a person type a message and see a reply. Whether you build on Open WebUI, your own single-page app, or something new, the bar is the same: it has to actually work end to end against your specialised model. A live stream processor: continuous automation The second capstone option swaps request-and-response for continuous automation: a stream processor that applies your model to an incoming feed as new items arrive, unattended, rather than waiting for a person to ask a question. A news-feed summariser is a natural example: new articles arrive, your pipeline calls the model to summarise or classify each one, and a quality gate (Module 07 and Module 08's shared discipline) validates each output before it is stored or forwarded. That last box matters more than it looks: a stream processor that runs for an hour against a live feed and then dies silently almost always failed because one malformed or unexpected input broke an unvalidated parsing step, the exact failure Module 07 taught you to guard against in a single call and that a long-running pipeline must guard against on every single item, indefinitely. Where you have arrived If your capstone produces all three things, a specialised expert model with a stated quality gate, a working chatbot, and a running stream processor, you have gone, concretely, from Module 00's perceptron to a self-hosted system you built, trained and run yourself. That was this course's whole promise, and it is now something you can point at and demonstrate, not just describe. Related CCI capabilities Computer Architecture (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/computer-architecture/). Optics Primer Series (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/optics/). Maths Refresher Series, Finance (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/maths-finance/). System Dynamics (Course): (https://www.cambridgecyberinternational.com/en/insights/academy/system-dynamics/). CCI Lab: Run it, build with it, read the thinking, reuse the data. (https://www.cambridgecyberinternational.com/en/insights/lab/)