Optimizing Quantum Model Performance for Professionals

Quantizing is not enough when fine-tuning a model! Even in the lowest precisions, most of the memory is going to be taken by the optimizer state when training that model! One great strategy that emerged recently is QLoRA. The idea is to apply LoRA adapters to quantized models. When the optimizer state is going to be computed, it is only going to be done on the adapter parameters instead of the whole model, and this will save a large amount of memory! The parameters are converted from BFloat16 / Float16 to 4-bits normal float. This quantization strategy comes from the realization that trained model weights tend to be Normal distributed, and we can create quantization buckets using that fact. This allows the compression of the model parameters without too much information loss. When we quantize a model, we need to capture the quantization constants to be able to dequantize the model. We usually capture them in Float32 to avoid as much dequantization error as possible. To compress further the model, we perform a double quantization to quantize the quantization constants to Float8. During the forward pass, because the input tensors are in BFloat16 / Float16, we need to dequantize the quantized parameters to perform the operations. However, during the backward pass, the original weights do not contribute to the computations, and they can remain quantized.

Quantum computing promises to making LLMs more efficient. And it's already working on real hardware. Efficient fine-tuning of large language models remains a critical bottleneck in AI development, with most researchers focused on purely classical computing approaches. A new paper from Chinese researchers demonstrates how quantum computing principles can dramatically reduce the parameters needed while improving model performance. The team introduces Quantum Weighted Tensor Hybrid Network (QWTHN), which combines quantum neural networks with tensor decomposition techniques to overcome the expressive limitations of traditional Low-Rank Adaptation (LoRA). By leveraging quantum state superposition and entanglement, their approach achieves remarkable efficiency: reducing trainable parameters by 76% while simultaneously improving performance by up to 15% on benchmark datasets. Most importantly, this isn't just theoretical - they've successfully implemented inference on actual quantum computing hardware. This represents a tangible advancement in making quantum computing practical for AI applications, demonstrating that even current-generation quantum devices can enhance the capabilities of billion-parameter language models. The integration of quantum techniques into traditional deep learning frameworks might become standard practice for resource-efficient AI development in the future. More on Quantum Hybrid Networks and other AI highlights in this week's LLM Watch:

Transform-to-Quantum: the problem with QUBO A recent pre-print from the QOBLIB team (“Quantum Optimization Benchmark Library -- The Intractable Decathlon” arXiv:2504.03832) provides a welcome trove of benchmark problems for quantum optimisation. Buried in Table 2 is a lesson every NISQ (or near term fault tolerant) project should pin to the wall: the way you model a problem can make or break any hope of running it on real hardware. Lets use the opportunity to highlight the “Transform to Quantum” part of our ITBQ Framework. Market Split—coefficients blow up • Mixed-Integer formulation: 78 binary variables, coefficient range ≈ 10². • After the routine MIP → QUBO conversion: still <100 variables, but the coefficient range balloons to ~7 × 10⁵. On an NISQ device that range must be encoded in gate angles or penalty weights. Six extra orders of magnitude usually translate into deeper circuits, worse conditioning and larger shot counts—effectively dooming a straightforward variational run. This is still similar on near term fault tolerant devices, in terms of the need for small angle rotations, which consume a substantial amount of T-Gates. LABS—variable count and coefficients explode • Original quadratic model: 81 spins, tiny coefficient range. • QUBO model: ~820 binary variables and a four-order-of-magnitude jump in coefficients. In most qubit-per-variable schemes that is a 10× increase in qubit demand plus the same precision nightmare seen in Market Split. Why it matters These examples are not corner cases—they are exactly the kind of “interesting, small” instances people reach for when chasing near-term quantum advantage. Yet without a smarter Transform-to-Quantum step (re-scaling, alternative encodings, constraint embedding, etc.) the numbers already exceed what today’s noisy processors can represent with meaningful fidelity. Take-away Hardware isn’t the only bottleneck. The modeling choices we make on the classical side determine whether a problem ever fits on quantum silicon. Getting Transform-to-Quantum right is therefore not a detail; it is the path-finder for every credible use case. Read more about our ITBQ Framework here: https://lnkd.in/en2KEjKC

✨ Introducing ruQu: Giving the quantum system an Ai voice. How I'm infusing self-learning/self-optimizing Ai with Quantum Machines. Today I've been exploring the more exotic underside of the intersection between quantum computing and self learning AI, and one pattern keeps showing up. We keep asking AI to push quantum hardware harder, faster, and further, without ever letting the hardware say when that pressure is unsafe. ruQu is a classical nervous system for quantum machines, and it unlocks a new class of AI infused quantum computing systems that were not viable before. Most AI plus quantum work treats AI as a tuner or optimizer. Adjust parameters. Improve decoders. Push performance. That assumes the quantum system is always safe to act on. In reality, quantum hardware is fragile. It drifts, it spikes, and blind optimization often accelerates failure instead of preventing it. ruQu changes that relationship. By measuring structural integrity in real time using boundary to boundary min cut, ruQu gives AI a sense of when the quantum system is healthy and when it is approaching breakage. That turns AI from an aggressive optimizer into a careful operator. It learns not just what to do, but when doing anything is a mistake. This enables a new class of systems where AI and quantum computing co evolve safely. Stable regions run fast. Fragile regions slow down or isolate. Learning pauses instead of corrupting state. The system behaves less like a brittle experiment and more like a living machine with reflexes. The real impact shows up at the application layer. In healthcare, this enables personalized quantum assisted diagnostics. Instead of running short, generic simulations, a system can run longer, patient specific models of protein folding, drug interactions, or genomic pathways without constant resets. That opens the door to customized treatment planning where each patient’s biology drives the computation, not the limitations of the hardware. In finance, it enables continuous risk modeling and stress testing that adapts in real time. Portfolio simulations can run longer and more safely, isolating instability instead of aborting entire analyses, which is critical for regulated environments that require auditability and reproducibility. AI infused quantum computing stops being fragile and opaque. It becomes segmented, self protecting, and operationally defensible. Integrity first. Then intelligence. Get started: Ask Claude to install: cargo add ruqu --features full  Or visit my RuVector Github repo. https://lnkd.in/gaB3gava

This week we made some big announcements about Qiskit. We officially launched our new vision for Qiskit as IBM’s software for mapping problems to circuits, optimizing circuits for quantum hardware, executing circuits on quantum hardware, and post-process their results. The expanded software stack of Qiskit now includes: - A faster (39x), more efficient (3x) Qiskit SDK for building, optimizing, and visualizing quantum circuits - An AI-enhanced Qiskit Transpiler Service giving 20-50% better CNOT count for typical circuits .  - Simplified execution modes (single, batch, dedicated sessions) for the Qiskit Runtime Service which can be tailored for performant (5x faster) execution of quantum circuits on quantum hardware. -  An updated Qiskit Serverless allowing you to run quantum-centric supercomputing workloads in the cloud. - Frictionless development with the Qiskit Code Assistant (released soon) to simplify code development. This is just the beginning of Qiskit performance improvements. Read more blog: https://lnkd.in/erYc7eQC white paper:  https://lnkd.in/eduiJpx2 Updated landing page: https://lnkd.in/ezeBfeJ9

I've been tackling the "barren plateaus" problem in QML, where training stalls inside vast search spaces. My latest experiment in fraud detection revealed a fascinating, counterintuitive solution. I discovered that increasing my quantum circuit's entanglement didn't smooth the path to a solution, but it created a more complex and rugged loss landscape (using a dressed quantum circuit scheme). Taking advantage of the hyvis library, I visualized this effect (thanks to the colleagues of JoS QUANTUM for putting this together), as shown in the first image of the post. The landscape evolves from a simple valley to a rich, expressive terrain (but potentially more complex for an optimizer). But did this complexity hurt performance? Usually that should be the case, but the exact opposite happened. The image shows the model with the most complex landscape (8 CNOTs by layer) not only learned faster (lower loss) but also achieved the highest accuracy (AUC) on the validation set and later in the test set. There is no free lunch on this. We can't generalize from these examples. This added complexity, or "expressivity," is precisely what allowed the model to find a superior solution in this case and avoid getting stuck, but it is not the norm. My biggest conclusion here It seems that for QML, the key to real-world performance isn't avoiding complexity, but leveraging it. To be able to extract permanent benefits, we should follow approaches like what Dra. Eva Andres Nuñez is researching by finding the way to use the extra complexity of entanglement to be able to find the global minima and not get stuck in our quantum optimization procedures using the theory behind SNNs. Here details about the hyvis library in GitHub: https://lnkd.in/dzqcFvDE An insightful paper from Eva about mixing SNNs and quantum: https://lnkd.in/dXDiuCBH Same subject from Jiechen Chen: https://lnkd.in/d-Uyngef #quantumcomputing #machinelearning #ai #datascience #frauddetection #ml #qml

First new paper of the year 🎉 : A new preprocessing technique to reduce QAOA circuit depth by 45% for real world problems. A major challenge in most applications of #quantumoptimization is that the resulting interdependencies of the variables are strongly coupled and dense. This results in deep quantum circuits that cannot be #reliably ran on quantum hardware. In this work, the #QGRID project team and collaborators developed a novel preprocessing algorithm to detect and transorm so-called “semi-symmetries” in QUBO matrix formulations, which increases the number of total qubits used while at the same time reducing the transpiled gate depth by up to 45% 💪 ‼️Paper link here 👉 https://lnkd.in/dtfcs7Mr Techniques such as this will be key to running 100-qubit scale optimizations for #realworld use-cases. Special thanks to Jonas Nüßlein and coauthors for this nice piece of work ! Leo Sünkel Jonas Stein Tobias Rohe Daniëlle Schuman Sebastian Feld Giorgio Cortiana Claudia Linnhoff-Popien E.ON Digital Technology Ludwig-Maximilians-Universität München Delft University of Technology #quantumcomputing #optimization #towardsquantumutility

QUANTUM INFORMATION GEOMETRY: QUANTUM NATURAL GRADIENT Review of Quantum Gradient Descent Algorithm and its use in different Quantum Optimization Algorithms Sangram Deshpande https://lnkd.in/eJ9WVNcu Abstract This review explores the quantum gradient descent algorithm, highlighting its distinctive advantages over classical gradient descent methods. Unlike the classical approach, which requires O(n) complexity for gradient computation, the quantum version achieves a remarkable complexity of O(1) through the principles of quantum superposition and entanglement. A particular focus is given to the Variational Quantum Eigensolver (VQE), a hybrid quantum-classical framework that utilizes parameterized quantum circuits combined with classical optimization routines. By replacing the classical optimizer with a quantum gradient descent optimizer, significant performance enhancements are realized, including faster convergence and reduced resource requirements. This work examines the implementation of quantum gradient methods within the VQE algorithm and emphasizes their potential to revolutionize optimization in quantum computing by providing exponential advantages in specific applications. The findings underscore the transformative role of quantum algorithms in advancing computational efficiency for complex optimization tasks. 

How can machine learning help us optimize results from today's noisy quantum computers? Error mitigation techniques like probabilistic error cancellation can recover accurate expectation values from noisy hardware, but at a high sampling cost which grows with the size and depth of the circuit. As we move toward utility-scale experiments, with hundreds of qubits and thousands of multi-qubit gates, this overhead becomes a true bottleneck. One idea is to use machine learning to reduce that cost. Instead of relying on additional circuit runs, you can train models to predict and correct for noise with data that has already been collected. In particular, we found that simple ML methods can meaningfully reduce the number of samples needed, while still producing accurate results. This approach sits at an interesting intersection of error mitigation and classical ML, and connects to a broader set of questions about how to make the most of the quantum hardware we have now. I covered a portion of this ML-QEM landscape, in a recent talk here: https://lnkd.in/gcVs8UjS How do you see these two areas intersecting as we move forward? #QuantumComputing #MachineLearning #ErrorMitigation #Physics #Science

Bridging Physics and AI: A Quantum-Inspired Leap in Neural Network Optimization I've mapped a transformer model (SmolLM2-135M) to an Ising spin system and achieved unprecedented optimization using Q*Agents - Heated Ballistic Bifurcation (Quantum Tunneling). What’s New? 🧠 Reimagining AI Optimization: I treated the transformer’s weights as a digital twin of a physical system, governed by quantum principles. Leveraged Q*agents in a quantum-inspired algorithm to reveal emergent structural patterns in neural networks. 📊 Results That Speak for Themselves: Chaotic weight distributions transformed into organized, periodic patterns. Quantum tunneling-like effects drove efficient optimization, even in high-dimensional spaces. Evidence of emergent organization—is this the physics of intelligence manifesting in AI? Why It Matters: 🔋 From Theory to Reality: Imagine training massive AI models with quantum efficiency—on classical hardware. These results pave the way for next-gen optimization techniques that blend physics and machine learning. ⚡ Key Technical Highlights: Model: SmolLM2-135M transformed via Ising spin mapping Method: Heated Ballistic Bifurcation Innovation: Emergent banding in weight matrices suggests fundamental principles shaping model architectures 🌌 This is only the beginning. Are physics-inspired approaches the next frontier in AI? Could this redefine how we design, train, and optimize large models? Let’s collaborate, discuss, and explore these exciting intersections. Share your insights or connect if you’re working in this space!