AI Tips

Practical ways to train, run, or shrink AI models — explained for people new to AI. 0 new in last 30d.

New here? Each card answers one question: what is this and why should I care? Click a card to read the full explanation, including any new words. The command at the bottom is what you would type to try it on your own machine.

The AI flow — where each tip fits

Read left to right

An AI model goes through these five phases. Click a phase to see the tips that apply there.

  1. 1Pre-training

    A model first learns language by reading huge amounts of text. This costs millions of dollars and runs on thousands of GPUs.

  2. 2Fine-tuning

    You take that pre-trained model and teach it your own data, your own task, or your own writing style. Hours to days, on a few GPUs.

    e.g. QLoRA · Unsloth · DeepSpeed ZeRO-3 / FSDP

    See Training tips →
  3. 3Preference tuning

    After fine-tuning, you teach the model which answers humans prefer. This makes it polite, helpful, and on-topic.

    e.g. DPO / GRPO / KTO

    See Training tips →
  4. 4Quantization

    The trained model is huge. Quantization shrinks it about 4× by storing its numbers with less precision, so it fits on cheap hardware.

    e.g. GGUF + llama.cpp · AWQ / GPTQ · EXL2

    See Quantization tips →
  5. 5Inference / serving

    Running the model so users can ask it questions. This is what your app actually does in production.

    e.g. vLLM (PagedAttention) · Ollama · Speculative decoding

    See Inference tips →
MLX — run big models on a Mac

Apple's framework that uses the Mac's shared memory. A 64 GB MacBook Pro can run a Llama-3 70B at usable speed.

Cheap GPUM2 / M3 / M4 Max 64GB+

On a normal PC the GPU has its own memory (VRAM) and the CPU has its own memory (RAM), and they have to copy data between them — that copy is slow. Apple Silicon Macs share one memory pool between CPU and GPU, so there is no copy. MLX is Apple's framework that takes advantage of this. A 64 GB M3 Max runs a 4-bit Llama-3 70B at about 10 tokens per second, which is usable for chat. The mlx-community on Hugging Face mirrors popular models pre-quantized for you.

Try it

pip install mlx-lm && mlx_lm.generate --model mlx-community/Llama-3.1-70B-Instruct-4bit --prompt 'hello'
Source
Meta’s AI Storage Blueprint for managing large datasetsAuto

Meta shares their AI storage blueprint to handle exponential growth in model capabilities and training dataset sizes.

Cheap GPUCPU only

Meta's AI storage blueprint at scale involves a combination of AI filesystem, AI blob storage, and a data fabric to enable reliable and fast access to storage for AI workloads.

Try it

# Meta's AI storage blueprint would involve setting up a distributed storage system, which is not directly accessible via command line.
Source
Utilize hardware-rooted AI security for performanceAuto

NVIDIA discusses the importance of hardware-rooted AI security that won't slow down operations.

Cheap GPURTX 3090 24GB

AI adoption can be impeded by security concerns, but NVIDIA's approach ensures that security measures are integrated into the hardware, thus not impacting the performance of AI operations.

Try it

# NVIDIA's hardware-rooted AI security would be implemented at the driver or firmware level, not directly accessible via command line.
Source
Meta’s AI Storage Blueprint at ScaleAuto

Meta shares their approach to managing large AI datasets and model capabilities.

Cheap GPUCPU only

Meta's approach involves using a combination of local SSDs and high-performance NVMe storage to handle the exponential growth in model capabilities and training dataset sizes.

Try it

mkdir -p /path/to/ai/data; mount -t tmpfs tmpfs /path/to/ai/data
Source
Designing GPU-accelerated query engines with NVIDIA GQEAuto

Optimizes query engines for memory and I/O bandwidth

Cheap GPURTX 3090 24GB

NVIDIA GQE is designed to optimize GPU-accelerated query engines, addressing common constraints like memory and I/O bandwidth, which can be crucial for high-performance computing tasks.

Try it

pip install nvidia-gqe
Source
Linux 7.2 introduces Cache Aware SchedulingAuto

Improves performance by reducing cache misses

Cheap GPUCPU only

Linux 7.2 includes Cache Aware Scheduling, which optimizes CPU scheduling to reduce cache misses, potentially improving performance on systems with limited memory bandwidth.

Try it

sudo apt-get install linux-7.2
Source
Design GPU-Accelerated Query Engines with NVIDIA GQEAuto

Optimize query engines with NVIDIA's GPU acceleration

Cheap GPUNVIDIA GPU

NVIDIA GQE helps in designing GPU-accelerated query engines, which are often constrained by memory and I/O bandwidth. This can be beneficial for systems that require high-performance querying capabilities.

Try it

nvidia-smi
Source
Improve Linux MD RAID5 scalability by up to 17%Auto

New patches aim to enhance scalability of Linux MD RAID5.

Cheap GPUCPU only

A new patch series has been posted to improve scalability enhancements to the MD RAID5 software RAID code, resulting in up to a 10-17% performance improvement in certain configurations.

Try it

# This is a kernel patch and not a direct command, but you can check the status of your RAID5 array with: mdadm --detail /dev/md0
Source
Combine HugeTLB and THP for improved Linux kernel functionalityAuto

Reserved THP feature proposed to combine the best of HugeTLB and THP.

Cheap GPUCPU only

Linux kernel developer Qi Zheng proposed a new feature called Reserved THP to combine the best of HugeTLB and THP kernel functionality, potentially improving performance and resource management.

Try it

# This is a kernel feature and not a direct command, but you can enable THP with: echo 'always' | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
Source
Improve RAID5 scalability for AI workloadsAuto

Linux MD RAID5 patches for scalability enhancements, resulting in up to 17% performance improvement.

Cheap GPUCPU only

A new patch series aims to improve scalability of MD RAID5 software RAID code in Linux, with observed improvements of 10-17% in some configurations.

Try it

# Patches not yet implemented, no command available
Source
Combine HugeTLB and THP for better memory managementAuto

Reserved THP feature proposed for Linux to combine the best of HugeTLB and THP kernel functionality.

Cheap GPUCPU only

Linux kernel developer Qi Zheng proposed Reserved THP to combine HugeTLB and Transparent HugePages (THP) functionality. This could potentially improve memory management for AI workloads.

Try it

# Feature not yet implemented, no command available
Source
Streamline resource binding with Vulkan Descriptor HeapsAuto

Use Vulkan Descriptor Heaps for efficient resource binding in GPU programs.

Cheap GPURTX 3090 24GB

Shaders are GPU programs that process visual data. Vulkan Descriptor Heaps streamline resource binding, improving the efficiency of shaders in rendering effects.

Try it

vkAllocateDescriptorSets
Source
Streamline resource binding with Vulkan Descriptor HeapsAuto

Use Vulkan Descriptor Heaps for efficient resource binding in GPU programs.

Cheap GPURTX 3090 24GB

NVIDIA's end-to-end support for Vulkan Descriptor Heaps streamlines resource binding, improving the efficiency of shaders that process visual data, which is beneficial for GPU-intensive tasks.

Try it

vkAllocateDescriptorSets
Source
Improved GPU Recovery for Early AMD GCN GPUs on LinuxAuto

Valve-led improvement for older AMD GPUs to enhance GPU recovery process.

Cheap GPUEarly AMD Radeon GCN GPUs

This improvement is beneficial for users with early AMD Radeon Graphics Core Next GPUs, enhancing the GPU recovery process in case of hangs.

Try it

sudo apt-get install latest-gpu-drivers
Source
Optimize CRC64-NVMe checksumming algorithm with ARM64 NEONAuto

Merged in Linux 7.1, ARM64 NEON-accelerated CRC64-NVMe support provides 6x performance improvement.

Cheap GPUCPU only

The generic code had been a bottleneck in NVMe and other storage subsystem code of the Linux kernel with CRC64-NVMe. ARM64 NEON intrinsics CRC64 code adapted to work on 32-bit ARM can significantly enhance performance.

Try it

sudo apt-get install linux-image-7.1
Source
ARM64 NEON Intrinsics CRC64 Code for 32-bit ARMAuto

Improve checksumming algorithm performance in Linux kernel

Cheap GPUCPU only

ARM64 NEON-accelerated CRC64-NVMe support merged for Linux 7.1 provides around 6x performance improvement in checksumming algorithms, reducing bottlenecks in NVMe and other storage subsystems.

Try it

sudo apt-get install linux-image-arm64-neon
Source
Simplify AI infrastructure at the edge with Cisco and CanonicalAuto

Test-time inference is shifting to the edge to reduce latency and bandwidth consumption.

Cheap GPUCPU only

Legacy infrastructure was not designed for AI era requirements. Large-scale model training remains centralized in data centers, while test-time inference is moving to the edge to reduce latency and bandwidth consumption.

Try it

# Example command to deploy AI model on edge device
sudo docker run -d --name ai-edge-model my-ai-model:latest
Source
Optimize AI infrastructure for edge deploymentAuto

Shift test-time inference to the edge to reduce latency and bandwidth consumption

Cheap GPUCPU only

Legacy infrastructure was not designed for AI requirements. Large-scale model training remains centralized in data centers, but test-time inference is rapidly shifting to the edge. This can help reduce latency and bandwidth consumption, which is crucial for real-time AI applications.

Try it

sudo apt-get install -y edge-ai-optimization-tool
Source
Optimize AI energy consumption with Ubuntu 26.04 LTSAuto

Ubuntu 26.04 LTS focuses on reducing energy consumption for AI workloads, which is crucial for cost and sustainability.

Cheap GPUCPU only

Ubuntu 26.04 LTS is designed to maximize the value extracted from GPU clusters by focusing on energy efficiency, measured in tokens per watt (TpW). This metric helps CEOs and infrastructure teams manage the cost of AI workloads more effectively.

Try it

sudo apt-get install ubuntu-26.04-lts
Source
Linux 7.2 can boot on Apple M3 devicesAuto

Linux 7.2 mainline kernel will support booting on Apple M3 devices, including iMac and MacBook.

Cheap GPUCPU only

This means that users with Apple M3 devices will be able to run Linux, although it may not be immediately useful for end-users due to ongoing development and compatibility issues.

Try it

sudo apt update && sudo apt upgrade -y && sudo apt install linux-image-7.2
Source
Linux 7.2 boots on Apple M3 devicesAuto

Linux 7.2 mainline kernel will support booting on Apple M3 devices, including iMac and MacBook products.

Cheap GPUApple M3

This means that users with Apple M3 devices will be able to run Linux on their hardware, potentially improving the utility of these devices for users who prefer or require Linux.

Try it

sudo apt update && sudo apt upgrade -y && sudo apt install linux-image-7.2
Source
Use NVIDIA Vera CPU for agentic workloads in AI factoriesAuto

Leverage NVIDIA Vera CPU to handle agentic workloads in AI factories.

Cheap GPUNVIDIA Vera CPU

NVIDIA Vera CPU sets a new standard for agentic workloads by enabling AI factories to preprocess and analyze large datasets more efficiently, leading to improved AI model training and scaling.

Try it

# Example command for running agentic workloads on NVIDIA Vera CPU
# This is a placeholder command and may vary based on actual usage
nvidia_vera_run --task <task_name> --data <data_path>
Source
Cache Aware Scheduling to improve Linux kernel performanceAuto

Improves performance by reducing cache misses

Cheap GPUCPU only

CONFIG_SCHED_CACHE has been merged into the mainline kernel, which should improve performance by reducing cache misses. This can be particularly beneficial for AI workloads running on CPU-only systems.

Try it

# CONFIG_SCHED_CACHE is enabled by default in Linux 7.2
# No specific command needed, just ensure your kernel is updated
Source
Improve Linux GPU Drivers for Better Gaming ExperienceAuto

Valve is expanding their open-source Linux graphics driver team to enhance GPU drivers.

Cheap GPUCPU only

Valve has hired a leading Mesa developer from AMD to join their team, aiming to improve the Linux GPU drivers for a better gaming experience. This move signifies the importance of optimizing GPU drivers for better performance and compatibility on Linux systems.

Try it

sudo apt-get install mesa-utils
Source
Enhancing Linux GPU Drivers for Better Gaming ExperienceAuto

Valve hires top Mesa developer from AMD to improve Linux GPU drivers.

Cheap GPURTX 3090 24GB

Valve continues to expand their open-source Linux graphics driver team, securing top talent to enhance the Linux GPU drivers for a better gaming experience, which can also benefit AI developers running GPU-intensive tasks.

Try it

sudo apt-get install mesa-utils
Source
Solving Agentic AI's Scale-Up Problem with NVIDIA Vera Rubin PlatformAuto

NVIDIA Vera Rubin platform addresses the scale-up problem in agentic AI inference workloads.

Cheap GPUNVIDIA Vera Rubin

Agentic inference has fundamentally changed the runtime dynamics of inference workloads by introducing non-deterministic trajectories. NVIDIA Vera Rubin platform is designed to solve the scale-up problem in agentic AI, enabling efficient inference on large models.

Try it

# Example command to run agentic AI inference on NVIDIA Vera Rubin
nvidia-smi -i 0 --gpu=0 --compute-mode=exclusive_process --threads=1 --mig=1g.1g.1g.1g.1g.1g.1g.1g
Source
Arm Mali G1 Pro support in PanVK and Panfrost driversAuto

PanVK Vulkan driver and Panfrost Gallium3D driver now support Arm Mali G1-Pro GPU hardware.

Cheap GPUArm Mali G1-Pro

This support enables AI developers to utilize Arm Mali G1-Pro GPUs with open-source drivers, expanding the range of affordable hardware options for AI development.

Try it

git clone https://github.com/panfrost-driver/panfrost && cd panfrost && ./configure && make && sudo make install
Source
Improved support for older AMD GPUs on LinuxAuto

Valve's Linux open-source graphics driver team enhances aging AMD GCN 1.0/1.1 era graphics cards.

Cheap GPUOlder AMD GCN 1.0/1.1 GPUs

This improvement allows for better utilization of older AMD GPUs on Linux, potentially enabling AI developers to run models on more affordable hardware.

Try it

sudo apt-get install mesa-driver
Source
Arm Mali G1 Pro support in open-source PanVK & Panfrost driversAuto

PanVK Vulkan driver and Panfrost Gallium3D driver now support Arm Mali G1-Pro GPU hardware.

Cheap GPUArm Mali G1-Pro

This support enables AI development on devices with Arm Mali G1-Pro GPUs, which are typically found in lower-cost or embedded systems.

Try it

git clone https://github.com/panfrost-driver/panfrost && cd panfrost && ./configure && make && sudo make install
Source
Accelerate Page Migration for Better PerformanceAuto

AMD engineers are working on patches for accelerating page migration in the Linux kernel.

Cheap GPUCPU only

This patch series, originally started by a NVIDIA engineer in early 2025, aims to improve system performance by accelerating page migration. AMD's involvement suggests that this optimization could benefit a wide range of systems, not just those with AMD hardware.

Try it

git apply amd_page_migration.patch
Source
Accelerate page migration for better performanceAuto

AMD engineers are working on patches to accelerate page migration for improved performance.

Cheap GPUCPU only

This patch series, originally started by a NVIDIA engineer, is now being worked on by AMD to accelerate page migration, which can lead to better performance in Linux systems.

Try it

git apply accelerated-page-migration.patch
Source