## Prerequisites and System Preparation

Before installing InstructLab, make sure your RHEL 9 system meets the requirements and has all necessary packages:

* **Operating System:** Red Hat Enterprise Linux 9 (64-bit). Update to the latest minor version (RHEL 9.2 or newer) so that newer Python versions are available.
* **Hardware:** At least 2 CPU cores, 8 GB RAM, and 70 GB of disk space (the default model download is \~4 GB, and fine-tuning workflows can use \~60 GB total). This guide assumes a non-GPU (CPU-only) environment.
* **Internet Access:** Ensure the VM can access the internet to download packages, models, and the taxonomy repository.

**System Packages:** Install development tools and language runtimes. RHEL 9’s default Python is 3.9, which is not sufficient for InstructLab (Python 3.10 or 3.11 is required, and Python 3.12 is *not* yet supported). We will install Python 3.11 and other build dependencies using `dnf`:

“`bash
# Update system (optional but recommended)
sudo dnf update -y

# Install required development tools and Python 3.11
sudo dnf install -y gcc gcc-c++ make cmake git python3.11 python3.11-devel rust cargo
“`

This command installs GCC (C/C++ compilers), Make, CMake, Git, **Python 3.11** (interpreter and dev headers), and the Rust toolchain (Rust is needed for building certain Python packages that use Rust/Cargo during installation). If `dnf` cannot find `python3.11`, ensure your system is updated to RHEL 9.2+ or enable the appropriate repositories (since RHEL 9.2 introduced a `python3.11` package). The Rust toolchain (`rustc` and `cargo`) is included via the `rust` and `cargo` packages; having these will prevent errors when pip compiles Rust-based dependencies.

**Note:** If you encounter an error about “Cargo (the Rust package manager) is not installed or on PATH” during pip install, it means a dependency requires a Rust compiler – ensure the Rust toolchain is installed and try again. Also, if you plan to use an **NVIDIA GPU** (not covered in this CPU-only guide), you would need to install CUDA drivers and toolkit, but for CPU-only setup you can ignore CUDA-related steps.

## Python Virtual Environment Setup

It’s recommended to use a Python virtual environment for the InstructLab installation to avoid conflicts with system packages. We will create a project directory and set up a Python 3.11 virtual environment:

“`bash
# Create and enter a workspace directory for InstructLab
mkdir ~/instructlab && cd ~/instructlab

# Create a Python 3.11 virtual environment (upgrade pip/setuptools inside)
python3.11 -m venv –upgrade-deps venv

# Activate the virtual environment
source venv/bin/activate

# Confirm Python version is 3.11
python –version # Should show Python 3.11.x
“`

This creates an isolated environment named `venv` in the `~/instructlab` directory and ensures the latest `pip` is available inside. All further commands in this guide assume the virtual environment is activated (the shell prompt should prefix with `(venv)`).

## Installing InstructLab (via pip)

The simplest way to install InstructLab is from PyPI using `pip`. For a CPU-only setup, InstructLab provides a special install option that includes the CPU-compatible dependencies (like the CPU version of PyTorch). We will use the `instructlab[cpu]` installation option:

“`bash
# (Ensure you are inside the activated Python venv)
# Optional: Clear any cached builds of llama_cpp_python to avoid reuse of old binaries
pip cache remove llama_cpp_python

# Install InstructLab with CPU-only support (no CUDA/GPU)
pip install ‘instructlab[cpu]’ \
–extra-index-url https://download.pytorch.org/whl/cpu \
-C cmake.args=”-DLLAMA_NATIVE=off”
“`

Let’s break down this command:

* **`instructlab[cpu]`:** Installs the InstructLab core package with extra dependencies for CPU operation. This pulls in the CPU-optimized **PyTorch** build and any other required libraries. (The extra index URL is the official PyTorch repository for CPU wheels.)
* **`-C cmake.args=”-DLLAMA_NATIVE=off”`:** This passes an option to the build of the `llama_cpp_python` backend to disable CPU-specific optimizations. It ensures compatibility with older CPUs that might not support certain advanced instructions (like `vpdpbusd`). If you skip this and encounter an *“unsupported instruction `vpdpbusd`”* error during install or runtime, reinstall with this flag to build a portable version of the llama.cpp backend.
* We removed any cached `llama_cpp_python` to force a fresh build with the new flags (this step is precautionary; it can be omitted on first-time installation).

This installation process may take some time as it will download and build multiple components (including compiling C++ code for the LLaMA backend). Once complete, verify that the `ilab` command is installed:

“`bash
ilab –help
“`

You should see the InstructLab CLI usage message (help listing available commands), confirming a successful install. For example, running `ilab` with no arguments should display a usage summary, including commands like `config`, `data`, `model`, `taxonomy`, etc..

## *Alternative:* Installing from Source (Git Clone + Poetry/Pip)

If you need the latest development version or want to contribute, you can install InstructLab from the source repository instead of PyPI:

1. **Clone the InstructLab repository**:

“`bash
git clone https://github.com/instructlab/instructlab.git
cd instructlab
“`

2. **Install with pip or Poetry**:

* **Using pip:** Ensure your virtual env is active, then install the package in editable mode (which also installs all dependencies):

“`bash
pip install -e .
“`

This will compile and install InstructLab from the local source.

* **Using Poetry:** If the project uses Poetry for dependency management, first install Poetry (`pip install poetry` if not already installed). Then run:

“`bash
poetry install
“`

This will create a virtualenv (or use the current one) and install all dependencies as defined in `pyproject.toml`. After installation, you may need to add the Poetry-created venv’s bin directory to your `PATH` or use `poetry run ilab …` to invoke the `ilab` CLI.

> **Note:** Source installation still requires the same prerequisites (Python 3.11, compilers, Rust, etc.). If the pip/poetry install fails due to missing system libraries or compilers, revisit the Prerequisites section.

## Initial Configuration and Model Setup

With InstructLab installed, you need to perform initial configuration. This will set up default directories and download the default model and taxonomy data used by InstructLab. The `ilab` CLI provides an interactive setup for this:

1. **Initialize InstructLab** by generating a default config and taxonomy:

“`bash
ilab config init
“`

This command will prompt you to configure the environment. For a typical local setup, you can accept the defaults:

* **Profile Defaults:** Press **Enter** when asked to provide values (this accepts default settings for directories and picks a system profile).
* **Clone Taxonomy:** When prompted *“Should I clone [https://github.com/instructlab/taxonomy.git](https://github.com/instructlab/taxonomy.git) for you?”*, type `y` and press Enter. This will download the official InstructLab **taxonomy** repository (containing sample knowledge/skills data) to the appropriate location.
* **Model Download:** Next, you’ll be prompted for a model path. Press **Enter** to accept the default path for the recommended model. If the model file isn’t found in that path, InstructLab will automatically download the default **Granite 7B** LLM model (in GGUF quantized format) for you. This download is a \~4–5 GB file, so it may take a while. Ensure you have sufficient disk space and a stable internet connection.

After these steps, the CLI should output a success message indicating initialization is complete. For example, you might see:

“`text
Generating config file: ~/.config/instructlab/config.yaml
…
Initialization completed successfully! You’re ready to start using `ilab`. Enjoy!
“`

InstructLab will have created a config file (by default under `~/.config/instructlab/config.yaml` or in your project directory) and set up local data folders for models and taxonomy. You can inspect the config file to see paths and settings.

2. **(Optional) Download a model manually:** If you skipped downloading a model during `config init` or want to fetch a specific model later, you can use:

“`bash
ilab model download
“`

By default this grabs the compact **Granite-7B-Lab** model (quantized in GGUF format) from Hugging Face. You can also specify other models or repositories with `–repository` and `–filename` options (see InstructLab docs for examples).

## Verifying the Installation

After initialization and model setup, verify that you can load and run the model:

* **Serve the Model:** Start the model inference server:

“`bash
ilab model serve
“`

This will launch a local server (default at `http://127.0.0.1:8000`) hosting the LLM. On a CPU-only setup, InstructLab uses the llama.cpp backend to run the model. You should see log output indicating the model was loaded, for example:

“`text
INFO … Using model ‘…/granite-7b-lab-Q4_K_M.gguf’ with -1 gpu-layers and 4096 max context size.
Starting server process…
After application startup complete, see http://127.0.0.1:8000/docs for API.
Press CTRL+C to shut down the server.
“`

. This means the model is now running and serving an API (compatible with OpenAI API schema) at port 8000.

* **Test a simple interaction:** In another terminal (with the virtual env activated), you can use the CLI to chat with the model:

“`bash
# Open an interactive chat session with the served model
ilab chat
“`

This will connect to the local model server and allow you to enter prompts. For example, try asking a question. If the model responds coherently, your installation is working. (Press `Ctrl+C` to exit the chat or stop the server as needed.)

*Alternatively*, you can test via the API: visit `http://127.0.0.1:8000/docs` in a browser to see the Swagger UI for the model’s REST API, or use `curl`/Postman to send a test query to `http://127.0.0.1:8000/v1/chat/completions`. For a quick test, however, the `ilab chat` CLI is easiest.

* **CLI commands:** You can also explore other CLI commands. For instance, `ilab model list` (if available) to list models, `ilab data generate` to generate synthetic data (requires a model running), etc. Use `ilab –help` or `ilab –help` for detailed usage.

If the model loads and you can interact with it, congratulations – InstructLab is successfully installed and configured on your RHEL 9 VM!

## Troubleshooting Common Issues on RHEL 9

* **Python Version Errors:** If you get errors during installation about an unsupported Python version, ensure you are using Python 3.10 or 3.11. Python 3.9 (RHEL’s default) will not work, and Python 3.12 is *not yet supported* by some InstructLab dependencies. Activate the correct Python binary (e.g., use `python3.11`) and consider recreating the virtual environment if needed.

* **Missing System Dependencies:** A failed build complaining about missing compilers, `pip` unable to build wheels, or errors like *“error: C++ compiler not found”* indicate you need to install development tools. Ensure you installed `gcc`, `gcc-c++`, `make`, and `cmake` via `dnf` as shown above. Similarly, if a Python package fails due to missing headers (e.g., Python.h), make sure `python3.11-devel` is installed.

* **Rust/Cargo Not Found:** If pip reports an error involving Cargo or Rust (for example, *“`cargo` not found”* or *“failed to build a rust extension”*), you need the Rust toolchain. Install Rust and Cargo as described in prerequisites (on RHEL you can use `dnf`, or use Rustup for the latest version). After installing Rust, re-run the `pip install …` command.

* **“Unsupported instruction `vpdpbusd`” Error:** This error can occur if the llama.cpp backend was compiled with CPU optimizations not supported by your processor. In our installation steps, we preemptively disabled those specific optimizations (`-DLLAMA_NATIVE=off`). If you did not include that and encounter an illegal instruction or similar error when running `ilab` or serving the model, reinstall InstructLab with the CMake flag to disable native optimizations. For example:

“`bash
pip install –no-cache-dir ‘instructlab[cpu]’ -C cmake.args=”-DLLAMA_NATIVE=off”
“`

This forces a rebuild without advanced CPU instructions.

* **RHEL Repositories and EPEL:** If `dnf` cannot find certain packages (like `rust` or `cmake`), you may need to enable additional repositories. On RHEL, ensure **CodeReady Builder** is enabled (for development packages) and consider enabling EPEL for supplemental packages. For example:

“`bash
sudo dnf config-manager –set-enabled crb
sudo dnf install -y cmake rust cargo
“`

This enables the CodeReady Builder repo (which contains compilers, CMake, etc. on RHEL) and then installs the missing packages. Always make sure your subscription or repositories are correctly set up for developer tooling.

* **Memory/Performance Issues:** Running a 7B model on CPU with 8 GB RAM is possible but can be slow. If you encounter out-of-memory errors when serving or training, try closing other processes to free memory. You can also create swap space as a fallback, or use a smaller model if available. Performance will not be fast on 2 CPU cores – expect inference to take several seconds (or more per response). This setup is suitable for functionality testing, but for serious use (or larger models) consider using a machine with more RAM or a GPU.

* **Firewall/Network:** The `ilab model serve` launches on port 8000. If you need to access this service from outside the VM, ensure that VMware network settings and RHEL’s firewall (`firewalld`) allow access to port 8000. For purely local testing (within the VM), this is not an issue.

* **Further Help:** Refer to the official InstructLab documentation and FAQ for more usage tips. The Red Hat Developer article *“Getting started with InstructLab for generative AI model tuning”* provides an overview and quick video demos. If you run into specific errors, searching the error message along with “InstructLab” often finds community solutions on GitHub or forums.

By following this guide, you should have a working InstructLab installation on your RHEL 9 VM, ready for you to experiment with fine-tuning and interacting with large language models.