Guide for installing InstructLab on a RHEL 9 virtual machine, optimized for a test environment.
I’ll walk through all necessary steps including prerequisites, Python environment setup, dependency installation, source code fetching, and initial usage.
Before installing InstructLab, make sure your RHEL 9 system meets the requirements and has all necessary packages:
System Packages: Install development tools and language runtimes. RHEL 9’s default Python is 3.9, which is not sufficient for InstructLab (Python 3.10 or 3.11 is required, and Python 3.12 is not yet supported). We will install Python 3.11 and other build dependencies using dnf:
# Update <span class="hljs-built_in">system</span> (optional but recommended)
sudo dnf <span class="hljs-keyword">update</span> -<span class="hljs-keyword">y</span>
# Install required development tools <span class="hljs-built_in">and</span> Python <span class="hljs-number">3.11</span>
sudo dnf install -<span class="hljs-keyword">y</span> gcc gcc-<span class="hljs-keyword">c</span>++ <span class="hljs-keyword">make</span> cmake git <span class="hljs-keyword">python3</span>.<span class="hljs-number">11</span> <span class="hljs-keyword">python3</span>.<span class="hljs-number">11</span>-devel rust cargo
This command installs GCC (C/C++ compilers), Make, CMake, Git, Python 3.11 (interpreter and dev headers), and the Rust toolchain (Rust is needed for building certain Python packages that use Rust/Cargo during installation). If dnf cannot find python3.11, ensure your system is updated to RHEL 9.2+ or enable the appropriate repositories (since RHEL 9.2 introduced a python3.11 package). The Rust toolchain (rustc and cargo) is included via the rust and cargo packages; having these will prevent errors when pip compiles Rust-based dependencies.
Note: If you encounter an error about “Cargo (the Rust package manager) is not installed or on PATH” during pip install, it means a dependency requires a Rust compiler – ensure the Rust toolchain is installed and try again. Also, if you plan to use an NVIDIA GPU (not covered in this CPU-only guide), you would need to install CUDA drivers and toolkit, but for CPU-only setup you can ignore CUDA-related steps.
It’s recommended to use a Python virtual environment for the InstructLab installation to avoid conflicts with system packages. We will create a project directory and set up a Python 3.11 virtual environment:
<span class="hljs-comment"># Create and enter a workspace directory for InstructLab</span>
mkdir ~<span class="hljs-regexp">/instructlab && cd ~/i</span>nstructlab
<span class="hljs-comment"># Create a Python 3.11 virtual environment (upgrade pip/setuptools inside)</span>
python3.<span class="hljs-number">11</span> -m venv --upgrade-deps venv
<span class="hljs-comment"># Activate the virtual environment</span>
source venv<span class="hljs-regexp">/bin/</span>activate
<span class="hljs-comment"># Confirm Python version is 3.11</span>
python --version <span class="hljs-comment"># Should show Python 3.11.x</span>
This creates an isolated environment named venv in the ~/instructlab directory and ensures the latest pip is available inside. All further commands in this guide assume the virtual environment is activated (the shell prompt should prefix with (venv)).
The simplest way to install InstructLab is from PyPI using pip. For a CPU-only setup, InstructLab provides a special install option that includes the CPU-compatible dependencies (like the CPU version of PyTorch). We will use the instructlab[cpu] installation option:
<span class="hljs-comment"># (Ensure you are inside the activated Python venv)</span>
<span class="hljs-comment"># Optional: Clear any cached builds of llama_cpp_python to avoid reuse of old binaries</span>
pip cache remove llama_cpp_python
<span class="hljs-comment"># Install InstructLab with CPU-only support (no CUDA/GPU)</span>
pip install <span class="hljs-string">'instructlab[cpu]'</span> \
--extra-index-url https:<span class="hljs-regexp">//</span>download.pytorch.org<span class="hljs-regexp">/whl/</span>cpu \
-C cmake.args=<span class="hljs-string">"-DLLAMA_NATIVE=off"</span>
Let’s break down this command:
instructlab[cpu]: Installs the InstructLab core package with extra dependencies for CPU operation. This pulls in the CPU-optimized PyTorch build and any other required libraries. (The extra index URL is the official PyTorch repository for CPU wheels.)-C cmake.args="-DLLAMA_NATIVE=off": This passes an option to the build of the llama_cpp_python backend to disable CPU-specific optimizations. It ensures compatibility with older CPUs that might not support certain advanced instructions (like vpdpbusd). If you skip this and encounter an “unsupported instruction vpdpbusd” error during install or runtime, reinstall with this flag to build a portable version of the llama.cpp backend.llama_cpp_python to force a fresh build with the new flags (this step is precautionary; it can be omitted on first-time installation).This installation process may take some time as it will download and build multiple components (including compiling C++ code for the LLaMA backend). Once complete, verify that the ilab command is installed:
ilab <span class="hljs-comment">--help</span>
You should see the InstructLab CLI usage message (help listing available commands), confirming a successful install. For example, running ilab with no arguments should display a usage summary, including commands like config, data, model, taxonomy, etc..
If you need the latest development version or want to contribute, you can install InstructLab from the source repository instead of PyPI:
Clone the InstructLab repository:
git <span class="hljs-keyword">clone</span> <span class="hljs-title">https</span>://github.com/instructlab/instructlab.git
cd instructlab
Install with pip or Poetry:
Using pip: Ensure your virtual env is active, then install the package in editable mode (which also installs all dependencies):
pip install <span class="hljs-_">-e</span> .
This will compile and install InstructLab from the local source.
Using Poetry: If the project uses Poetry for dependency management, first install Poetry (pip install poetry if not already installed). Then run:
poetry <span class="hljs-keyword">install</span>
This will create a virtualenv (or use the current one) and install all dependencies as defined in pyproject.toml. After installation, you may need to add the Poetry-created venv’s bin directory to your PATH or use poetry run ilab ... to invoke the ilab CLI.
Note: Source installation still requires the same prerequisites (Python 3.11, compilers, Rust, etc.). If the pip/poetry install fails due to missing system libraries or compilers, revisit the Prerequisites section.
With InstructLab installed, you need to perform initial configuration. This will set up default directories and download the default model and taxonomy data used by InstructLab. The ilab CLI provides an interactive setup for this:
Initialize InstructLab by generating a default config and taxonomy:
ilab <span class="hljs-built_in">config</span> init
This command will prompt you to configure the environment. For a typical local setup, you can accept the defaults:
y and press Enter. This will download the official InstructLab taxonomy repository (containing sample knowledge/skills data) to the appropriate location.After these steps, the CLI should output a success message indicating initialization is complete. For example, you might see:
Generating <span class="hljs-built_in">config</span> file: ~/.<span class="hljs-built_in">config</span>/instructlab/<span class="hljs-built_in">config</span>.yaml
...
Initialization completed successfully! You<span class="hljs-string">'re ready to start using `ilab`. Enjoy!</span>
InstructLab will have created a config file (by default under ~/.config/instructlab/config.yaml or in your project directory) and set up local data folders for models and taxonomy. You can inspect the config file to see paths and settings.
(Optional) Download a model manually: If you skipped downloading a model during config init or want to fetch a specific model later, you can use:
<span class="hljs-attribute">ilab model download</span>
By default this grabs the compact Granite-7B-Lab model (quantized in GGUF format) from Hugging Face. You can also specify other models or repositories with --repository and --filename options (see InstructLab docs for examples).
After initialization and model setup, verify that you can load and run the model:
Serve the Model: Start the model inference server:
<span class="hljs-attribute">ilab model serve</span>
This will launch a local server (default at http://127.0.0.1:8000) hosting the LLM. On a CPU-only setup, InstructLab uses the llama.cpp backend to run the model. You should see log output indicating the model was loaded, for example:
<span class="hljs-function"><span class="hljs-title">INFO</span></span> ... <span class="hljs-keyword">Using</span> <span class="hljs-keyword">model</span> <span class="hljs-string">'.../granite-7b-lab-Q4_K_M.gguf'</span> with <span class="hljs-number">-1</span> gpu-layers <span class="hljs-keyword">and</span> <span class="hljs-number">4096</span> <span class="hljs-built-in">max</span> context size.
<span class="hljs-function"><span class="hljs-title">Starting</span></span> server process...
After application startup complete, see http:<span class="hljs-comment">//127.0.0.1:8000/docs for API.</span>
Press CTRL+C to shut down the server.
. This means the model is now running and serving an API (compatible with OpenAI API schema) at port 8000.
Test a simple interaction: In another terminal (with the virtual env activated), you can use the CLI to chat with the model:
<span class="hljs-comment"># Open an interactive chat session with the served model</span>
<span class="hljs-attribute">ilab</span> chat
This will connect to the local model server and allow you to enter prompts. For example, try asking a question. If the model responds coherently, your installation is working. (Press Ctrl+C to exit the chat or stop the server as needed.)
Alternatively, you can test via the API: visit http://127.0.0.1:8000/docs in a browser to see the Swagger UI for the model’s REST API, or use curl/Postman to send a test query to http://127.0.0.1:8000/v1/chat/completions. For a quick test, however, the ilab chat CLI is easiest.
CLI commands: You can also explore other CLI commands. For instance, ilab model list (if available) to list models, ilab data generate to generate synthetic data (requires a model running), etc. Use ilab --help or ilab <command> --help for detailed usage.
If the model loads and you can interact with it, congratulations – InstructLab is successfully installed and configured on your RHEL 9 VM!
Python Version Errors: If you get errors during installation about an unsupported Python version, ensure you are using Python 3.10 or 3.11. Python 3.9 (RHEL’s default) will not work, and Python 3.12 is not yet supported by some InstructLab dependencies. Activate the correct Python binary (e.g., use python3.11) and consider recreating the virtual environment if needed.
Missing System Dependencies: A failed build complaining about missing compilers, pip unable to build wheels, or errors like “error: C++ compiler not found” indicate you need to install development tools. Ensure you installed gcc, gcc-c++, make, and cmake via dnf as shown above. Similarly, if a Python package fails due to missing headers (e.g., Python.h), make sure python3.11-devel is installed.
Rust/Cargo Not Found: If pip reports an error involving Cargo or Rust (for example, “cargo not found” or “failed to build a rust extension”), you need the Rust toolchain. Install Rust and Cargo as described in prerequisites (on RHEL you can use dnf, or use Rustup for the latest version). After installing Rust, re-run the pip install ... command.
“Unsupported instruction vpdpbusd” Error: This error can occur if the llama.cpp backend was compiled with CPU optimizations not supported by your processor. In our installation steps, we preemptively disabled those specific optimizations (-DLLAMA_NATIVE=off). If you did not include that and encounter an illegal instruction or similar error when running ilab or serving the model, reinstall InstructLab with the CMake flag to disable native optimizations. For example:
<span class="hljs-attribute">pip</span> install --<span class="hljs-literal">no</span>-cache-dir <span class="hljs-string">'instructlab[cpu]'</span> -C cmake.args=<span class="hljs-string">"-DLLAMA_NATIVE=off"</span>
This forces a rebuild without advanced CPU instructions.
RHEL Repositories and EPEL: If dnf cannot find certain packages (like rust or cmake), you may need to enable additional repositories. On RHEL, ensure CodeReady Builder is enabled (for development packages) and consider enabling EPEL for supplemental packages. For example:
sudo dnf config-manager --<span class="hljs-keyword">set</span>-enabled <span class="hljs-comment">crb</span>
sudo <span class="hljs-comment">dnf install -y cmake rust cargo</span>
This enables the CodeReady Builder repo (which contains compilers, CMake, etc. on RHEL) and then installs the missing packages. Always make sure your subscription or repositories are correctly set up for developer tooling.
Memory/Performance Issues: Running a 7B model on CPU with 8 GB RAM is possible but can be slow. If you encounter out-of-memory errors when serving or training, try closing other processes to free memory. You can also create swap space as a fallback, or use a smaller model if available. Performance will not be fast on 2 CPU cores – expect inference to take several seconds (or more per response). This setup is suitable for functionality testing, but for serious use (or larger models) consider using a machine with more RAM or a GPU.
Firewall/Network: The ilab model serve launches on port 8000. If you need to access this service from outside the VM, ensure that VMware network settings and RHEL’s firewall (firewalld) allow access to port 8000. For purely local testing (within the VM), this is not an issue.
Further Help: Refer to the official InstructLab documentation and FAQ for more usage tips. The Red Hat Developer article “Getting started with InstructLab for generative AI model tuning” provides an overview and quick video demos. If you run into specific errors, searching the error message along with “InstructLab” often finds community solutions on GitHub or forums.
By following this guide, you should have a working InstructLab installation on your RHEL 9 VM, ready for you to experiment with fine-tuning and interacting with large language models. Enjoy building with InstructLab!