Transformers RISC-V

没想到，转换成gguf格式，需要变形金刚。
使用 pytorch 运行大语言模型，需要用到 transformers，但是 RISC-V 没有。
先用 llama.cpp 试试。

使用花里胡哨前端网站 hugging face拉取你最喜爱的大预言模型吧！
安装完 python-transformers, python-pytorch, 发现pacman里没有 sentencepiece。
快来一起使用有毒的 python 虚拟环境吧！

python -m venv ~/.venv/sentencepiece
source ~/.venv/sentencepiece/bin/activate
pip install sentencepiece pytorch transformers

pytorch 下了一堆，令人害怕。
成功搭建起可以使用 llama.cpp 中的 convert_hf_to_gguf.py 的 python 虚拟环境将大语言模型转换成 .gguf 格式之后，把 .gguf 文件传到 K1 上，成功跑起来了三秒蹦出一个字的大语言模型。
当然，以 K1 的性能不能按照默认配置跑。

Loading model... \[1]    1812866 killed     ./build/bin/llama-cli --model ../deepseek-r1-distill-qwen-1.5b.gguf -p

需要修改配置：

./build/bin/llama-cli \
  -m ../deepseek-r1-distill-qwen-1.5b.gguf \
  -p "你好" \
  -c 1024 \ // 降低 KV cache
  --no-mmap \ // 避免页缓存 + OOM
  --mlock \ // 防止被 swap
  -t 1 // 单线程

但是只用 llama.cpp 跑是不行的，需要用 pytorch 。

在 K1 上用 python 虚拟环境可以下载 sentencepiece 但是不能用 transformers。
再打个包试试。ArchLinux 的 gitlab 不让我 https 拉包了 Orz，申请账号ing。
然而搞笑的是，https 拉不到包是因为 ArchLinux 官方仓库里本来就没有这个包。因为我没有看搜到包的来源是 archlinuxcn，不过就算看了也不是知道属于 AUR。

native

尝试从源码在 RISC-V 上构建 transformers。

        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.13-64bit" PYO3_PYTHON="/home/xyenchi/.venv/transformers/bin/python" PYTHON_SYS_EXECUTABLE="/home/xyenchi/.venv/transformers/bin/python" "cargo" "rustc" "--profile" "release" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/pip-install-pjykn0yj/tokenizers_88c57405900f4e5c8e26fca9b20a80ef/bindings/python/Cargo.toml" "--lib"`
      Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/home/xyenchi/.venv/transformers/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
  Building wheel for safetensors (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for safetensors (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [297 lines of output]

        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.13-64bit" PYO3_PYTHON="/home/xyenchi/.venv/transformers/bin/python" PYTHON_SYS_EXECUTABLE="/home/xyenchi/.venv/transformers/bin/python" "cargo" "rustc" "--profile" "release" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/pip-install-pjykn0yj/safetensors_eeec8ab73bd1467fba4b87a6d7305d04/bindings/python/Cargo.toml" "--lib"`
      Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/home/xyenchi/.venv/transformers/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for safetensors
Successfully built transformers
Failed to build tokenizers safetensors
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> tokenizers, safetensors

safetensor 需要 python版本<=3.12

error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)
  = help: please check if an updated version of PyO3 is available. Current version: 0.21.1
  = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.13-64bit" PYO3_PYTHON="/usr/bin/python" PYTHON_SYS_EXECUTABLE="/usr/bin/python" "cargo" "rustc" "--profile" "release" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/build/python-safetensors/src/safetensors-0.4.3/bindings/python/Cargo.toml" "--lib"`
Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/bin/python', '--compatibility', 'off'] returned non-zero exit status 1

终于给 safetensors 和 tokenizers 都设置环境变量 export PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 并给 tokenizers 配置环境变量静态链接之后好了 Orz。但也因此错失了入坑 nix 的机会，没有吃上安利真是可惜。

真亏我把 python-regex python-huggingface_hub python_transformers python-safetensors python-tokenizers 打了个遍。

native#

native