An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference

Abstract

Cloud LLM services typically require users to send prompts to a model provider, creating a privacy risk. Fully homomorphic encryption (FHE) lets a server perform inference without decrypting the input, but representing data as ciphertexts adds storage and computational overhead. In CKKS-based LLM inference, the packing scheme maps logical tensors to ciphertexts and slots. It therefore determines the ciphertext count and the homomorphic cost of linear layers, and it constrains how data pass betwe

Categories

Framework mappings

Suggested from the entry's categories.

@misc{fan2026opensource,
  title = {{An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference}},
  author = {Yuhang Fan and Yusi Chen and Kanyu Ye and Zhuoran Ji},
  year = {2026},
  month = sep,
  eprint = {2609.12378},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.12378}
}