Strategies for Optimizing Models

The best way to tune a Large Language Model depends on what you need it to do. Improving instruction-following, building domain expertise, reducing training costs, and aligning responses with specific expectations all require different approaches. That is why it is important to understand the fine-tuning techniques available. Here are 20 methods worth knowing: → 𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿-𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗺𝗲𝘁𝗵𝗼𝗱𝘀 LoRA, QLoRA, Prefix Tuning, Adapter Tuning, P-Tuning, BitFit, and Soft Prompts adapt a model without updating all its parameters. These methods are useful when computing power, memory, training data, or time is limited. → 𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗽𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁 Instruction Tuning improves the model’s ability to understand and follow user commands. RLHF, RLAIF, DPO, GRPO, and RLVR use human feedback, AI-generated feedback, preference pairs, group-based rewards, or verifiable outcomes to improve behaviour. → 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗮𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻 Continued Pretraining and Domain-Adaptive Pretraining expose the model to specialised data relevant to a particular industry or use case. This can improve performance in areas such as finance, healthcare, legal workflows, engineering, and customer support. → 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗼𝗽𝘁𝗶𝗺𝗶𝘀𝗮𝘁𝗶𝗼𝗻 Multi-Task Tuning, Federated Tuning, Data Selection, and Curriculum Learning can improve generalisation, privacy, efficiency, and training stability. → 𝗙𝘂𝗹𝗹 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 This approach updates every parameter in the model. It offers maximum flexibility but requires significantly more data, memory, computing power, and training time. The right technique depends on: ↳ The task you are solving ↳ The quality of your training data ↳ The size of the model ↳ Your available computing budget ↳ The level of control you need For many teams, LoRA, QLoRA, or Instruction Tuning is the most practical place to begin. Which fine-tuning technique would you choose for your current use case?

Not all errors are equal. Some are worth fixing more than others. Imagine you’re building a model to predict customer churn. A false negative—predicting a customer will stay when they actually leave—can cost thousands of dollars in lost revenue. A false positive—predicting churn when the customer would have stayed—might only cost a small retention offer. Treating these mistakes as equal, like most accuracy metrics do, misses the bigger picture. This is where 𝐜𝐨𝐬𝐭-𝐬𝐞𝐧𝐬𝐢𝐭𝐢𝐯𝐞 𝐦𝐨𝐝𝐞𝐥𝐢𝐧𝐠 comes in. Instead of optimizing for raw accuracy, you can tell your model which mistakes are more costly. In practice, this can be done by: 👉 Weighted loss functions: Modify your training loss to penalize false negatives more than false positives. For example, if using logistic regression or neural networks, you can apply class weights in cross-entropy loss. 👉 Resampling techniques: Oversample the minority “high-cost” class (in this case, churners) or undersample low-cost classes to bias the model towards minimizing high-impact mistakes. Even a well-trained model needs careful 𝐭𝐡𝐫𝐞𝐬𝐡𝐨𝐥𝐝 𝐭𝐮𝐧𝐢𝐧𝐠. The default 0.5 probability isn’t always optimal. You can: 👉 Use business-driven thresholds: Choose the cutoff that maximizes expected revenue or minimizes cost based on your confusion matrix. 👉 Perform grid search or optimization over thresholds using your validation set and the monetary cost associated with each type of prediction. Another way to approach this is through 𝐞𝐱𝐩𝐞𝐜𝐭𝐞𝐝 𝐯𝐚𝐥𝐮𝐞 𝐦𝐨𝐝𝐞𝐥𝐢𝐧𝐠. Assign a real-world cost or gain to each type of prediction, compute the net expected gain over your validation set, and tune the model or threshold to maximize that. This moves the focus from “statistical correctness” to business impact. 𝐔𝐧𝐜𝐞𝐫𝐭𝐚𝐢𝐧𝐭𝐲 also matters. High-confidence predictions are usually reliable, but when the model is unsure—like a probability near 0.5—you can: 👉 Flag these for human review. 👉 Use ensembles or Bayesian models to quantify uncertainty and guide intervention strategies. Finally, don’t forget 𝐦𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠 𝐚𝐟𝐭𝐞𝐫 𝐝𝐞𝐩𝐥𝐨𝐲𝐦𝐞𝐧𝐭. The business environment changes, and so do the costs associated with errors. Regularly recalibrating your thresholds and retraining models ensures you continue focusing on the mistakes that matter most. The key takeaway: chasing perfect accuracy is rarely the goal. By understanding which errors are costly, adjusting your model to focus on them, and incorporating uncertainty into decisions, you build models that not only predict but actually deliver measurable business value.

𝐈𝐬 𝐘𝐨𝐮𝐫 𝐌𝐨𝐝𝐞𝐥 𝐀𝐜𝐜𝐮𝐫𝐚𝐭𝐞 𝐛𝐮𝐭 𝐓𝐨𝐨 𝐁𝐢𝐠, 𝐒𝐥𝐨𝐰, 𝐨𝐫 𝐄𝐱𝐩𝐞𝐧𝐬𝐢𝐯𝐞 𝐭𝐨 𝐀𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐃𝐞𝐩𝐥𝐨𝐲? Training a great model is half the job. Getting it into production affordably and fast, that's the other half. Three compression techniques, three different approaches to the same problem. 1. Pruning : Trim the Unnecessary • Remove neurons or connections that don't contribute much to predictions. • Like gardening less clutter, faster computation. • Same architecture, just lighter and leaner. • Payoff: faster inference, lower compute cost, minimal accuracy impact. • The model keeps its structure but loses the parts that weren't pulling their weight. 2. Knowledge Distillation : Teach a Smaller Model • A large "teacher" model trains a smaller "student" to mimic its behavior. • Like mentorship, the student learns the essential patterns without all the complexity. • The teacher's soft targets during training help the student capture knowledge in a fraction of the parameters. • Payoff: much smaller model, faster inference, retained high performance. • You end up with a different, smaller model that behaves like the big one. 3. Quantization : Use Fewer Bits • Store numbers with lower precision, 32-bit floats down to 8-bit integers. • Like measuring with a slightly coarser ruler, good enough, and much faster. • Massively reduces memory use and boosts speed. • Payoff: tiny precision loss for a big efficiency win. • The model stays the same structurally, just represented with smaller numbers. How do they actually differ? • Pruning removes parts of the network. • Distillation transfers knowledge into a smaller network. • Quantization shrinks the precision of the numbers themselves. When do you use each? • Model has redundant connections: Prune. • Need a fundamentally smaller model that performs like the big one: Distill. • Need faster inference with minimal changes: Quantize. • Need maximum compression: Stack all three together. They're not mutually exclusive. In practice, production teams often prune first, distill into a smaller architecture, then quantize for deployment. Each technique compounds the gains of the others. Knowing which lever to pull and in which order is what gets your model from notebook to production. Which technique made the biggest difference in your deployment? ♻️ Repost this to help your network get started ➕ Follow Sivasankar Natarajan for more #ModelCompression #DeepLearning #AIEngineering

Quantizing is not enough when fine-tuning a model! Even in the lowest precisions, most of the memory is going to be taken by the optimizer state when training that model! One great strategy that emerged recently is QLoRA. The idea is to apply LoRA adapters to quantized models. When the optimizer state is going to be computed, it is only going to be done on the adapter parameters instead of the whole model, and this will save a large amount of memory! The parameters are converted from BFloat16 / Float16 to 4-bits normal float. This quantization strategy comes from the realization that trained model weights tend to be Normal distributed, and we can create quantization buckets using that fact. This allows the compression of the model parameters without too much information loss. When we quantize a model, we need to capture the quantization constants to be able to dequantize the model. We usually capture them in Float32 to avoid as much dequantization error as possible. To compress further the model, we perform a double quantization to quantize the quantization constants to Float8. During the forward pass, because the input tensors are in BFloat16 / Float16, we need to dequantize the quantized parameters to perform the operations. However, during the backward pass, the original weights do not contribute to the computations, and they can remain quantized.

Most people evaluate LLMs by just benchmarks. But in production, the real question is- how well do they perform? When you’re running inference at scale, these are the 3 performance metrics that matter most: 1️⃣ Latency How fast does the model respond after receiving a prompt? There are two kinds to care about: → First-token latency: Time to start generating a response → End-to-end latency: Time to generate the full response Latency directly impacts UX for chat, speed for agentic workflows, and runtime cost for batch jobs. Even small delays add up fast at scale. 2️⃣ Context Window How much information can the model remember- both from the prompt and prior turns? This affects long-form summarization, RAG, and agent memory. Models range from: → GPT-3.5 / LLaMA 2: 4k–8k tokens → GPT-4 / Claude 2: 32k–200k tokens → GPT-OSS-120B: 131k tokens Larger context enables richer workflows but comes with tradeoffs: slower inference and higher compute cost. Use compression techniques like attention sink or sliding windows to get more out of your context window. 3️⃣ Throughput How many tokens or requests can the model handle per second? This is key when you’re serving thousands of requests or processing large document batches. Higher throughput = faster completion and lower cost. How to optimize based on your use case: → Real-time chat or tool use → prioritize low latency → Long documents or RAG → prioritize large context window → Agentic workflows → find a balance between latency and context → Async or high-volume processing → prioritize high throughput My 2 cents 🤌 → Choose in-region, lightweight models for lower latency → Use 32k+ context models only when necessary → Mix long-context models with fast first-token latency for agents → Optimize batch size and decoding strategy to maximize throughput Don’t just pick a model based on benchmarks. Pick the right tradeoffs for your workload. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

If you’re leading AI initiatives, here is a strategic cheat sheet to move from "𝗰𝗼𝗼𝗹 𝗱𝗲𝗺𝗼" to 𝗲𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝘃𝗮𝗹𝘂𝗲. Think Risk, ROI, and Scalability. This strategy moves you from "𝘄𝗲 𝗵𝗮𝘃𝗲 𝗮 𝗺𝗼𝗱𝗲𝗹" to "𝘄𝗲 𝗵𝗮𝘃𝗲 𝗮 𝗯𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗮𝘀𝘀𝗲𝘁." 𝟭. 𝗧𝗵𝗲 "𝗪𝗵𝘆" 𝗚𝗮𝘁𝗲 (𝗣𝗿𝗲-𝗣𝗼𝗖) • Don’t build just because you can. Define the Business Problem first • Success: Is the potential value > 10x the estimated cost? • Decision: If the problem can be solved with Regex or SQL, kill the AI project now. 𝟮. 𝗧𝗵𝗲 𝗣𝗿𝗼𝗼𝗳 𝗼𝗳 𝗖𝗼𝗻𝗰𝗲𝗽𝘁 (𝗣𝗼𝗖) • Goal: Prove feasibility, not scalability. • Timebox: 4–6 weeks max. • Team: 1-2 AI Engineers + 1 Domain Expert (Data Scientist alone is not enough). • Metric: Technical feasibility (e.g., "Can the model actually predict X with >80% accuracy on historical data?") 𝟯. 𝗧𝗵𝗲 "𝗠𝗩𝗣" 𝗧𝗿𝗮𝗻𝘀𝗶𝘁𝗶𝗼𝗻 (𝗧𝗵𝗲 𝗩𝗮𝗹𝗹𝗲𝘆 𝗼𝗳 𝗗𝗲𝗮𝘁𝗵) • Shift from "Notebook" to "System." • Infrastructure: Move off local GPUs to a dev cloud environment. Containerize. • Data Pipeline: Replace manual CSV dumps with automated data ingestion. • Decision: Does the model work on new, unseen data? If accuracy drops >10%, halt and investigate "Data Drift." 𝟰. 𝗥𝗶𝘀𝗸 & 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 (𝗧𝗵𝗲 "𝗟𝗮𝘄𝘆𝗲𝗿" 𝗣𝗵𝗮𝘀𝗲) • Compliance is not an afterthought. • Guardrails: Implement checks to prevent hallucination or toxic output (e.g., NeMo Guardrails, Guidance). • Risk Decision: What is the cost of a wrong answer? If high (e.g., medical advice), keep a "Human-in-the-Loop." 𝟱. 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 • Scalability & Latency: Users won’t wait 10 seconds for a token. • Serving: Use optimized inference engines (vLLM, TGI, Triton) • Cost Control: Implement token limits and caching. "Pay-as-you-go" can bankrupt you overnight if an API loop goes rogue. 𝟲. 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 • Automated Eval: Use "LLM-as-a-Judge" to score outputs against a golden dataset. • Feedback Loops: Build a mechanism for users to Thumbs Up/Down outcomes. Gold for fine-tuning later. 𝟳. 𝗢𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝘀 (𝗟𝗟𝗠𝗢𝗽𝘀) • Day 2 is harder than Day 1. • Observability: Trace chains and monitor latency/cost per request (LangSmith, Arize). • Retraining: Models rot. Define when to retrain (e.g., "When accuracy drops below 85%" or "Monthly"). 𝗧𝗲𝗮𝗺 𝗘𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 • PoC Phase: AI Engineer + Subject Matter Expert. • MVP Phase: + Data Engineer + Backend Engineer. • Production Phase: + MLOps Engineer + Product Manager + Legal/Compliance. 𝗛𝗼𝘄 𝘁𝗼 𝗺𝗮𝗻𝗮𝗴𝗲 𝗔𝗜 𝗣𝗿𝗼𝗷𝗲𝗰𝘁𝘀 (𝗺𝘆 𝗮𝗱𝘃𝗶𝗰𝗲): → Treat AI as a Product, not a Research Project. → Fail fast: A failed PoC cost $10k; a failed Production rollout costs $1M+. → Cost Modeling: Estimate inference costs at peak scale before you write a line of production code. What decision gates do you use in your AI roadmap? Follow Priyanka for more cloud and AI tips and tools #ai #aiforbusiness #aileadership

Benchmarks? More like benchwarmers! amiright? Over the last four months, I've spent 200+ hours playing with open-source models on HuggingFace. And I've found that, while benchmarks are decent signal, they don't always translate into practical effectiveness or correlate with how I'm planning on using a model. So, I want to pull back the curtain and share my 'vibe check' method because I don't like blindly following leaderboard rankings. 🏁 Starting with Baseline Generations What I Do: I test 10-15 diverse prompts using the model's default generation parameters. Why It Matters: This step gives me a raw, unfiltered look at the model's out-of-the-box behaviour and sets a baseline for further experimentation. ✅ Selective Prompt Analysis Process: I choose a balanced mix of 3-5 prompts, some showcasing the model's high performance and others where it falls short. Objective: It's all about levelling the playing field. This way, I get to see the real impact of tweaking those parameters. Just straightforward insights into how these changes play out. 🎛️ Parameter Adjustment - One at a Time Approach: I experiment with one parameter at a time — temperature, num_beams, top_k, top_p, repetition_penalty, no_repeat_ngram_size. Goal: Observing changes in output helps me understand how each parameter influences the model’s responses. At this point I usually have hundreds of generations from the model. 🕵️♂️ Deep Dive into Model Behavior Method: I manually review the generations, hunting for odd or undesirable outputs. Insight: This granular analysis is crucial for identifying the model's subtle nuances and potential pitfalls. 💻 Writing Targeted Tests: Strategy: Develop tests for specific issues noticed during the exploratory phase (e.g., output length, gibberish, repetition). Use type-token ratio for assessing lexical diversity, and check for repeat n-gram sizes. Purpose: Makes it easier to do more fine-grained statistical analysis down the line. 🧩 The Grid Search: Execution: I perform a detailed grid search over a range of parameter values across several prompts. Aim: Find a handful of effective settings that consistently yield desirable results. 🎯 The Final Stretch: Process: I test these top settings across an expanded set of 20+ prompts, looking for consistent performance and reliability. Result: This gives me a comprehensive understanding of how the model behaves under various settings and prompts. 🔬 Utilizing Advanced Tools: Integration: Finally, I use tools like LangChain's criteria evaluators with GPT-4 to assess output. Benefit: This step adds a layer of sophistication and accuracy to the selection process. 🫵🏽 Your Turn: Share Your Insights! Do you have a different approach to selecting and tuning LLMs? Share your strategies, tips, or even constructive critiques. Looking forward to your stories and experiences in the comments below! #generativeai #llm #opensource

Good folks at Apple have developed a novel method called KV Prediction that significantly reduces the "time to first token" (TTFT) for on-device LLM inference. Some highlights of the paper: • Uses a small auxiliary transformer model to efficiently predict the KV cache of a larger base model • Reduces TTFT by up to 4x while retaining 60-80% accuracy on benchmarks • Achieves Pareto-optimal efficiency-accuracy trade-off compared to baselines • Demonstrates 15-50% relative accuracy improvements on TriviaQA at equal TTFT FLOP budgets • Shows up to 30% accuracy gains on HumanEval code completion at fixed TTFT FLOP counts • Validated on Apple M2 Pro CPU, proving FLOP gains translate to real-world speedups So, how's it done? Based on the KV Prediction method described in the paper, here are the key steps for how it's done: 1. Choose a base model and an auxiliary model: - The base model is a larger, pretrained transformer model that will be used for final generation. - The auxiliary model is a smaller transformer model used to efficiently process the input prompt. 2. Design the KV predictor: - Create a set of learned linear projections to map from the auxiliary model's KV cache to the base model's KV cache. - Define a mapping from auxiliary cache layers to base cache layers. 3. Training process: - Pass input tokens through the auxiliary model to get its KV cache. - Use the KV predictor to generate a predicted KV cache for the base model. - Run the base model using the predicted KV cache and compute losses. - Backpropagate errors through the frozen base model to update the auxiliary model and KV predictor. 4. Inference process: - Process the input prompt with the auxiliary model to get its KV cache. - Use the KV predictor to generate the predicted base model KV cache. - Run a single token generation step with the base model using the predicted KV cache. - Continue autoregressive generation with the base model as normal. 5. Optimize the model: - Experiment with different auxiliary model sizes and architectures. - Tune the loss function, including base loss, auxiliary loss, and consistency loss components. - Evaluate efficiency-accuracy trade-offs on relevant benchmarks. Excited to hear your thoughts! What other applications do you see for this technique?

After optimizing costs for many AI systems, I've developed a systematic approach that consistently delivers cost reductions of 60-80%. Here's my playbook, in order of least to most effort: Step 1: Optimizing Inference Throughput Start here for the biggest wins with least effort. Enabling caching (LiteLLM (YC W23), Zilliz) and strategic batch processing can reduce costs by a lot with very little effort. I have seen teams cut costs by half simply by implementing caching and batching requests that don't require real-time results. Step 2: Maximizing Token Efficiency This can give you an additional 50% cost savings. Prompt engineering, automated compression (ScaleDown), and structured outputs can cut token usage without sacrificing quality. Small changes in how you craft prompts can lead to massive savings at scale. Step 3: Model Orchestration Use routers and cascades to send prompts to the cheapest and most effective model for that prompt (OpenRouter, Martian). Why use GPT-4 for simple classification when GPT-3.5 will do? Smart routing ensures you're not overpaying for intelligence you don't need. Step 4: Self-Hosting I only suggest self-hosting for teams at scale because of the complexities involved. This requires more technical investment upfront but pays dividends for high-volume applications. The key is tackling these layers systematically. Most teams jump straight to self-hosting or model switching, but the real savings come from optimizing throughput and token efficiency first. What's your experience with AI cost optimization?

Thought you know all about DP? Here’s an expanded tour of DP optimization techniques, from the fundamentals all the way to advanced tricks: 1. Top-Down vs. Bottom-Up 🔹 Memoization (recursion + cache) 🔹 Tabulation (iterative table filling) 2. Space-Saving Strategies 🔹 Rolling arrays: Keep only the last one or two rows (or dimensions) of your DP table. 🔹 Bitsets: Pack small states into bit operations for ultra-fast transitions. 3. Prefix-Sum & Difference Tricks 🔹 Precompute cumulative sums to reduce O(N) transition loops to O(1). 🔹 Use difference arrays for range-update patterns in DP. 4. Monotonic Queue / Sliding Window 🔹 For “min/max over last K states” problems, maintain a deque of candidates in amortized O(1) per update. 5. Bitmask & SOS-DP 🔹 Bitmask DP for subsets of up to ~20 elements (2ⁿ states). 🔹 SOS (Sum Over Subsets) DP to compute functions on all subsets via fast zeta transforms. 6. Segment-Tree-Backed DP 🔹 Use a segment tree (or Fenwick tree) to answer range min/max queries or do range updates on your DP array in O(log N). 🔹 Merge DP states efficiently when you need non-trivial transitions over intervals. 7. 1D/1D (Monge or Quadrangle-Inequality) Optimization 🔹 Targets recurrences of the form dp[i] = min_{0 ≤ j < i} [dp[j] + w(j, i)] where w satisfies the quadrangle (Monge) inequality, so the argmin indices k(i) are non-decreasing. 🔹 Use divide-and-conquer to compute all dp[i] in O(N log N), or Knuth’s optimization to push it to O(N) when stronger conditions hold . 8. Divide-and-Conquer Optimization 🔹 A special case of 1D/1D when optimal split points are monotonic: drop O(N²) down to O(N log N) by recursively solving on segments and narrowing search ranges. 9. Knuth / Quadrangle Inequality 🔹 When cost functions satisfy the quadrangle inequality and boundary conditions, you can reduce range-DP from O(N³) to O(N²) (or even to O(N) in certain forms). 10. Convex Hull Trick & Li Chao Tree 🔹 Optimize linear recurrences of the form dp[i] = min_j [m_j * x_i + b_j] from O(N²) to O(N log N) (or O(N) with a monotonic hull). 11. FFT-Based Convolution 🔹 Use fast polynomial multiplication (FFT) to merge DP steps in O(N log N) instead of O(N²). 12. Matrix Exponentiation / Chain Exponentiation 🔹 Model linear recurrences as dp_vec[i] = M * dp_vec[i−1] Raise the transition matrix M to the nᵗʰ power in O(k³ log n) (or faster) to compute dp[n] in logarithmic time. 13. Berlekamp–Massey Algorithm 🔹 Given the first 2k terms of a sequence, extract its minimal linear recurrence in O(k²). 🔹 Combine with fast exponentiation to compute the nᵗʰ term in O(k² log n), even for very large n. 14. Slope Trick & Aliens’ Tricks 🔹 Handle piecewise-linear DP functions and complex cost updates by maintaining envelopes of slopes. 🔹 Ideal for “add a V-shaped penalty” or “minimize sum of absolute deviations plus a quadratic cost.” Mastering these tools will raise your problem-solving skills, whether you’re in a contest or a interview.