Paper 2026/1551

Fine-Grained and Runtime-Configurable Precision for Exact FHE Inference

Abstract

Privacy-preserving machine learning under fully homomorphic encryption (FHE) faces a structural limitation: numerical precision is bound to cryptographic parameters and key material, forcing precision to be fixed at scheme initialization. Existing frameworks must regenerate keys or recompile circuits whenever bit-width changes, eliminating precision as a deployment-time performance knob and making mixed-precision strategies - widely used in plaintext machine learning - impractical under encryption. We present $\mathtt{Tailor}$, a backend-agnostic framework for exact, runtime-configurable mixed-precision neural inference over bitwise FHE. By representing signed integers as vectors of independently encrypted bits and constructing all arithmetic and neural-network operators from precision-parametric Boolean circuits, $\mathtt{Tailor}$ decouples bit-width from cryptographic state. Per-layer precision becomes a runtime parameter under a single keygen, and nonlinear operators, including ReLU, absolute value, and comparisons, are evaluated exactly rather than via polynomial approximation. A fused saturating requantize - ReLU produces compact unsigned activations, and each accumulator is provisioned at its provably minimal width. A $\mathtt{Scheme<Backend>}$ abstraction makes the framework portable across bitwise FHE families; we instantiate it for both TFHE and FINAL, along with a plaintext reference backend for functional validation and exact gate accounting. Our evaluation of an MNIST classifier shows that mixed-precision configurations strictly Pareto-dominate uniform deployment. Reducing only the output-layer weights to 2 bits while keeping inputs, activations, and hidden-layer weights at 4 bits yields the highest accuracy among all evaluated settings, 96.44% in 222.0s. This is both more accurate and faster than uniform 4-bit inference (96.17% in 236.1s) and $2.8\times$ faster than uniform 8-bit (96.14% in 632.4s). Reducing the hidden-layer weights to 2 bits instead trades 0.4 accuracy points for a $1.5\times$ speedup over uniform 4-bit (95.80% in 158.2s).