6 phút đọc
gufo-org/gufo: The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg – Repository gufo-org/gufo. Mô tả: The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2. 958 stars, 159 forks, ngôn ngữ chính C++, cập nhật 2026-10-27.
1. Repository làm gì?
Gufo is a vertical local inference engine for Linux and Windows, specifically built and optimized for AMD Strix Halo hardware: Ryzen AI MAX+ 395 systems with Radeon 8060S (gfx1151), up to 128 GiB of unified memory.
See the changelog and GitHub Releases for user-facing changes and release history.
Check out Gufo RDMA for Dual Strix Halo, a Gufo variant that has not been merged yet!
All model documentation lives under docs/models:
Peak measured workloads; text pp is autoregressive (AR), while tg uses the named speculative mode. Peaks include repetitive output; aggregate tg sums individual request decode rates. Qwen27B's single-user peak uses the short-prompt C1 workload. Audio excludes loading. Each model guide lists the required files and complete benchmark settings.
Rootless Podman needs crun for –group-add keep-groups. Your host user must have read/write access to /dev/kfd and /dev/dri/renderD*, usually through the render and video groups; log out and back in after changing membership. Container groups named video/render do not preserve host supplementary groups. Check id, ls -l /dev/kfd /dev/dri/renderD*, and podman info –format '{{.Host.OCIRuntime.Name}}' if ROCm reports no device. See Podman's rootless group-access guidance.
2. Dữ liệu và cấu trúc đáng chú ý
On Fedora or another SELinux-enforcing host, GPU enumeration can succeed while SELinux blocks mapping /dev/kfd, causing ROCr to report a misleading “Memory critical” error. Check the host audit log:
If it shows a denied map for the container, Podman documents this fix:
This persistently allows containers to access device labels for devices passed into them; it affects all containers on that host. Review that policy scope before enabling it. See Podman's device documentation and the SELinux container policy.
Then, from another terminal, ask it something through the OpenAI-compatible API:
For Open WebUI, VS Code, OpenAI SDK and Responses-API clients (for example Codex), set the API base URL to http://localhost:8080/v1. Chat Completions and Responses support text, images, function tools, structured output and streaming. See the API contract.
The text server uses the model's native context by default and generates until EOS or the context is full. –context N sets context capacity per session; –max-tokens N sets a default response limit that clients can override. Reasoning tokens count toward that response limit.
Production packages target Linux x86-64 on AMD Strix Halo (gfx1151). Windows x86-64 is a supported native build target for development; see BUILD-WINDOWS.md. CMake owns one production configuration for both Nix and ordinary Linux builds. Tests, profilers, tuning executables and Python reference runners are not installed with the production package. No build.sh wrapper is needed.
3. Khả năng ứng dụng và giới hạn
flake.lock pins the dependencies. nix develop adds profiling, model-download and independent evaluation tools; these are not runtime requirements. Optional benchmark baselines are selected separately with nix shell .#ds4-reference, .#llama-cpp-reference or .#llama-cpp-mtp-reference; see benchmarking. See testing for the small hosted CI suite and explicit local quality checks.
Prebuilt store paths are published to the gufo Cachix cache. The flake declares it in nixConfig: users listed in trusted-users are asked to accept it, or pass –accept-flake-config. Nix ignores it for other users; add it system-wide with cachix use gufo or in /etc/nix/nix.conf:
The Nix package and development shell use ROCm 10.1.0 (HIP Clang 24) from the pinned nixpkgs ROCm 10.1 update. Python, Python packages and general build dependencies come from the pinned nixos-unstable input. The ROCm package set comes from the separate fork, retaining its compiler/runtime dependencies to reuse the gfx1151 binaries on gufo.cachix.org. All Nix workflows configure this cache explicitly. Both include a queue-memory patch for Strix Halo. It keeps queue save/restore buffers, launch/copy arenas and code-loading staging resident during host memory compaction, while retaining managed-memory support and the ordinary fine-grained host-allocation policy.
Install a C++20 compiler, CMake 3.21+, Ninja, pkg-config and the following development libraries. The pinned Nix toolchain uses GCC 16.2 and ROCm 10.1.0 (HIP Clang 24). Attention and audio convolution kernels are compiled directly from HIP. Python, Triton/AOTriton, Composable Kernel and MIOpen are not production build or runtime requirements. For the same protection against queue evictions as the Nix build, apply the ROCr patch above when building your runtime. Flash-Next's host buffers also use Linux long-term pins, with ordinary HIP registration retained when system limits disallow those pins. Gufo leaves HSA_USERPTR_FOR_PAGED_MEM unchanged.
Install ROCm using AMD's Linux instructions. Use the development packages for the libraries above. ROCm normally installs under /opt/rocm.
For example, on Debian/Ubuntu the ordinary system libraries are:
Configuring fails at find_package(hipblas) when those ROCm packages are missing. Other distributions name them -devel instead of -dev. For nonstandard installations, pass ordinary CMake paths, for example cmake –preset release -DCMAKE_PREFIX_PATH="/opt/rocm". If compiler discovery picks a system Clang, also pass -DCMAKE_HIP_COMPILER=/opt/rocm/llvm/bin/clang++. Use cmake –install build/release to install Gufo, its runtime data and license notices. The GPU driver must allow your user to access /dev/kfd and /dev/dri; model weights are acquired separately.
The same source, compiler flags and install rules serve both builds. Nix pins the complete toolchain for reproducible comparisons; changing the compiler or math libraries requires the affected model's quality checks.
Gufo's original code is MIT licensed. Adapted code and dependencies retain their own notices in NOTICE, THIRD_PARTY_NOTICES.md and licenses/, installed under share/licenses/gufo. Model weights are not bundled and retain their publishers' terms.
Số liệu repository có thể thay đổi; nên mở liên kết nguồn để kiểm tra README, giấy phép và trạng thái phát hành.
Nguồn tham khảo: Xem bài gốc
