Using LLMs to Solve Workflow Bottlenecks

I’ve been building and managing data systems at Amazon for the last 8 years. Now that AI is everywhere, the way we work as data engineers is changing fast. Here are 5 real ways I (and many in the industry) use LLMs to work smarter every day as a Senior Data Engineer: 1. Code Review and Refactoring LLMs help break down complex pull requests into simple summaries, making it easier to review changes across big codebases. They can also identify anti-patterns in PySpark, SQL, and Airflow code, helping you catch bugs or risky logic before it lands in prod. If you’re refactoring old code, LLMs can point out where your abstractions are weak or naming is inconsistent, so your codebase stays cleaner as it grows. 2. Debugging Data Pipelines When Spark jobs fail or SQL breaks in production, LLMs help translate ugly error logs into plain English. They can suggest troubleshooting steps or highlight what part of the pipeline to inspect next, helping you zero in on root causes faster. If you’re stuck on a recurring error, LLMs can propose code-level changes or optimizations you might have missed. 3. Documentation and Knowledge Sharing Turning notebooks, scripts, or undocumented DAGs into clear internal docs is much easier with LLMs. They can help structure your explanations, highlight the “why” behind key design choices, and make onboarding or handover notes quick to produce. Keeping platform wikis and technical documentation up to date becomes much less of a chore. 4. Data Modeling and Architecture Decisions When you’re designing schemas, deciding on partitioning, or picking between technologies (like Delta, Iceberg, or Hudi), LLMs can offer quick pros/cons, highlight trade-offs, and provide code samples. If you need to visualize a pipeline or architecture, LLMs can help you draft Mermaid or PlantUML diagrams for clearer communication with stakeholders. 5. Cross-Team Communication When collaborating with PMs, analytics, or infra teams, LLMs help you draft clear, focused updates, whether it’s a Slack message, an email, or a JIRA comment. They’re useful for summarizing complex issues, outlining next steps, or translating technical decisions into language that business partners understand. LLMs won’t replace data engineers, but they’re rapidly raising the bar for what you can deliver each week. Start by picking one recurring pain point in your workflow, then see how an LLM can speed it up. This is the new table stakes for staying sharp as a data engineer.

LLMs are the single fastest way to make yourself indispensable and give your team a 30‑percent productivity lift. Here is the playbook. Build a personal use‑case portfolio Write down every recurring task you handle for clients or leaders: competitive intelligence searches, slide creation, meeting notes, spreadsheet error checks, first‑draft emails. Rank each task by time cost and by the impact of getting it right. Start automating the items that score high on both. Use a five‑part prompt template Role, goal, context, constraints, output format. Example: “You are a procurement analyst. Goal: draft a one‑page cost‑takeout plan. Context: we spend 2.7 million dollars on cloud services across three vendors. Constraint: plain language, one paragraph max. Output: executive‑ready paragraph followed by a five‑row table.” Break big work into a chain of steps Ask first for an outline, then for section drafts, then for a fact‑check. Steering at each checkpoint slashes hallucinations and keeps the job on‑track. Blend the model with your existing tools Paste the draft into Excel and let the model write formulas, then pivot. Drop a JSON answer straight into Power BI. Send the polished paragraph into PowerPoint. The goal is a finished asset, not just a wall of text. Feed the model your secret sauce Provide redacted samples of winning proposals, your slide master, and your company style guide. The model starts producing work that matches your tone and formatting in minutes. Measure the gain and tell the story Track minutes saved per task, revision cycles avoided, and client feedback. Show your manager that a former one‑hour job now takes fifteen minutes and needs one rewrite instead of three. Data beats anecdotes. Teach the team Run a ten‑minute demo in your weekly stand‑up. Share your best prompts in a Teams channel. Encourage colleagues to post successes and blockers. When the whole team levels up, you become known as the catalyst, not the cost‑cutting target. If every person on your team gained back one full day each week, what breakthrough innovation would you finally have the bandwidth to launch? What cost savings could you achieve? What additional market share could you gain?

Building LLM Agent Architectures on AWS - The Future of Scalable AI Workflows What if you could design AI agents that not only think but also collaborate, route tasks, and refine results automatically? That’s exactly what AWS’s LLM Agent Architecture enables. By combining Amazon Bedrock, AWS Lambda, and external APIs, developers can build intelligent, distributed agent systems that mirror human-like reasoning and decision-making. These are not just chatbots - they’re autonomous, orchestrated systems that handle workflows across industries, from customer service to logistics. Here’s a breakdown of the core patterns powering modern LLM agents : Breakdown: Key Patterns for AI Workflows on AWS 1. Prompt Chaining / Saga Pattern Each step’s output becomes the next input — enabling multi-step reasoning and transactional workflows like order handling, payments, and shipping. Think of it as a conversational assembly line. 2. Routing / Dynamic Dispatch Pattern Uses an intent router to direct queries to the right tool, model, or API. Just like a call center routing customers to the right department — but automated. 3. Parallelization / Scatter-Gather Pattern Agents perform tasks in parallel Lambda functions, then aggregate responses for efficiency and faster decisions. Multiple agents think together — one answer, many minds. 4. Saga / Orchestration Pattern Central orchestrator agents manage multiple collaborators, synchronizing tasks across APIs, data sources, and LLMs. Perfect for managing complex, multi-agent projects like report generation or dynamic workflows. 5. Evaluator / Reflect-Refine Loop Pattern Introduces a feedback mechanism where one agent evaluates another’s output for accuracy and consistency. Essential for building trustworthy, self-improving AI systems. AWS enables modular, event-driven, and autonomous AI architectures, where each pattern represents a step toward self-reliant, production-grade intelligence. From prompt chaining to reflective feedback loops, these blueprints are reshaping how enterprises deploy scalable LLM agents. #AIAgents

I started by asking AI to do everything. Six months later, 65% of my agent’s workflow nodes run as non-AI code. The first version was fully agentic : every task went to an LLM. LLMs would confidently progress through tasks, though not always accurately. So I added tools to constrain what the LLM could call. Limited its ability to deviate. I added a Discovery tool to help the AI find those tools. Better, but not enough. Then I found Stripe’s minion architecture. Their insight : deterministic code handles the predictable ; LLMs tackle the ambiguous. I implemented blueprints, workflow charts written in code. Each blueprint specifies nodes, transitions between them, trigger conditions for matching tasks, & explicit error handling. This differs from skills or prompts. A skill tells the LLM what to do. A blueprint tells the system when to involve the LLM at all. Each blueprint is a directed graph of nodes. Nodes come in two types : deterministic (code) & agentic (LLM). Transitions between nodes can branch based on conditions. Deal pipeline updates, chat messages, & email routing account for 29% of workflows, all without a single LLM call. Company research, newsletter processing, & person research need the LLM for extraction & synthesis only. Another 36%. The workflow runs 67-91% as code. The LLM sees only what it needs : a chunk of text to summarize, a list to categorize, processed in one to three turns with constrained tools. Blog posts, document analysis, bug fixes are genuinely hybrid. 21% of workflows. Multiple LLM calls iterate toward quality. Only 14% remain fully agentic. Data transforms & error investigations. These tend to be coding tasks rather than evaluating a decision point in a workflow. The LLM needs freedom to explore. AI started doing everything. Now it handles routing, exceptions, research, planning, & coding. The rest runs without it. Is AI doing less? Yes. Is the system doing more? Also yes. The blueprints, the tools, the skills might be temporary scaffolding. With each new model release, capabilities expand. Tasks that required deterministic code six months ago might not tomorrow.

We know LLMs can substantially improve developer productivity. But the outcomes are not consistent. An extensive research review uncovers specific lessons on how best to use LLMs to amplify developer outcomes. 💡 Leverage LLMs for Improved Productivity. LLMs enable programmers to accomplish tasks faster, with studies reporting up to a 30% reduction in task completion times for routine coding activities. In one study, users completed 20% more tasks using LLM assistance compared to manual coding alone. However, these gains vary based on task complexity and user expertise; for complex tasks, time spent understanding LLM responses can offset productivity improvements. Tailored training can help users maximize these advantages. 🧠 Encourage Prompt Experimentation for Better Outputs. LLMs respond variably to phrasing and context, with studies showing that elaborated prompts led to 50% higher response accuracy compared to single-shot queries. For instance, users who refined prompts by breaking tasks into subtasks achieved superior outputs in 68% of cases. Organizations can build libraries of optimized prompts to standardize and enhance LLM usage across teams. 🔍 Balance LLM Use with Manual Effort. A hybrid approach—blending LLM responses with manual coding—was shown to improve solution quality in 75% of observed cases. For example, users often relied on LLMs to handle repetitive debugging tasks while manually reviewing complex algorithmic code. This strategy not only reduces cognitive load but also helps maintain the accuracy and reliability of final outputs. 📊 Tailor Metrics to Evaluate Human-AI Synergy. Metrics such as task completion rates, error counts, and code review times reveal the tangible impacts of LLMs. Studies found that LLM-assisted teams completed 25% more projects with 40% fewer errors compared to traditional methods. Pre- and post-test evaluations of users' learning showed a 30% improvement in conceptual understanding when LLMs were used effectively, highlighting the need for consistent performance benchmarking. 🚧 Mitigate Risks in LLM Use for Security. LLMs can inadvertently generate insecure code, with 20% of outputs in one study containing vulnerabilities like unchecked user inputs. However, when paired with automated code review tools, error rates dropped by 35%. To reduce risks, developers should combine LLMs with rigorous testing protocols and ensure their prompts explicitly address security considerations. 💡 Rethink Learning with LLMs. While LLMs improved learning outcomes in tasks requiring code comprehension by 32%, they sometimes hindered manual coding skill development, as seen in studies where post-LLM groups performed worse in syntax-based assessments. Educators can mitigate this by integrating LLMs into assignments that focus on problem-solving while requiring manual coding for foundational skills, ensuring balanced learning trajectories. Link to paper in comments.

Sad but true. The closer the agent gets to production, the more the cracks begin to show. Teams make two mistakes at this point. They either look for a bigger/smarter LLM or endlessly iterate on prompting and RAG. Neither works. Successful agents start with the workflow, not the LLM. The more detailed the description of the workflow and outcomes, the less the agent needs to rely on AI. Every time the agent must guess what the next step is or what tools and information to use at this step, it creates an opportunity for small mistakes. They compound across multiple steps into much larger failures. Next, the workflow and the domain expertise required to deliver the outcome must be built into a knowledge graph. Trying to stuff everything into markdown files is a recipe for hallucination pie. The longer the file, the harder it is for LLMs to keep things straight. They lose focus and lose sight of what information is important. Knowledge graphs fix this by giving the agent exactly the information it needs at exactly the step it needs it. When agents get lost, and uncertainty metrics rise, the knowledge graph can deliver examples and metrics that define success and refocus the agent on iterating until it builds an acceptable output. Knowledge graphs can deliver guardrails that prevent agents from falling into endless loops. The goal is to build agents that rely on LLMs as little as possible and only deploy LLMs for what they are good at. Use the smallest models possible, and open-source models should handle over 80% of the workflow. Finally, agents need real-world feedback to improve. Version 1 is never perfect, and it takes multiple improvement cycles to be ready for deployment. Agents and knowledge graphs must be architected to benefit from improvement cycles. Every mistake creates the data required to ensure it never happens again.

Focusing on AI’s hype might cost your company millions… (Here’s what you’re overlooking) Every week, new AI tools grab attention—whether it’s copilot assistants or image generators. While helpful, these often overshadow the true economic driver for most companies: AI automation. AI automation uses LLM-powered solutions to handle tedious, knowledge-rich back-office tasks that drain resources. It may not be as eye-catching as image or video generation, but it’s where real enterprise value will be created in the near term. Consider ChatGPT: at its core, there is a large language model (LLM) like GPT-3 or GPT-4, designed to be a helpful assistant. However, these same models can be fine-tuned to perform a variety of tasks, from translating text to routing emails, extracting data, and more. The key is their versatility. By leveraging custom LLMs for complex automations, you unlock possibilities that weren’t possible before. Tasks like looking up information, routing data, extracting insights, and answering basic questions can all be automated using LLMs, freeing up employees and generating ROI on your GenAI investment. Starting with internal process automation is a smart way to build AI capabilities, resolve issues, and track ROI before external deployment. As infrastructure becomes easier to manage and costs decrease, the potential for AI automation continues to grow. For business leaders, identifying bottlenecks that are tedious for employees and prone to errors is the first step. Then, apply LLMs and AI solutions to streamline these operations. Remember, LLMs go beyond text—they can be used in voice, image recognition, and more. For example, Ushur is using LLMs to extract information from medical documents and feed it into backend systems efficiently—a task that was historically difficult for traditional AI systems. (Link in comments) In closing, while flashy AI demos capture attention, real productivity gains come from automating tedious tasks. This is a straightforward way to see returns on your GenAI investment and justify it to your executive team.

𝐈 𝐜𝐮𝐭 𝐦𝐲 𝐋𝐋𝐌 𝐩𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐭𝐢𝐦𝐞 𝐛𝐲 𝟏𝟒𝐱 𝐰𝐢𝐭𝐡 𝐨𝐧𝐞 𝐬𝐢𝐦𝐩𝐥𝐞 𝐜𝐡𝐚𝐧𝐠𝐞 I just witnessed the power of parallel processing with LLMs, and the results are too good not to share. Last month, I posted a video demonstrating three approaches to running LLM calls efficiently in n8n. https://lnkd.in/exJ6SdrC The third method – parallel calls to the same Basic LLM chain – delivered significant time savings on a real-world project. I needed to categorize a catalog of items with unstructured text descriptions, a perfect test case. The numbers speak for themselves. Using Gemini-2.5 Flash, each individual request averaged 3 seconds (ranging from 2.5 to 8.5 seconds per item). Running these sequentially would have taken around 4 minutes. With parallel processing? The entire batch completed in under 18 seconds. 𝐓𝐡𝐚𝐭'𝐬 𝐚𝐥𝐦𝐨𝐬𝐭 𝟏𝟒𝐱 𝐬𝐩𝐞𝐞𝐝 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭! While my case was relatively small, imagine this same optimization applied to thousands of items. The difference between minutes and seconds adds up quickly at scale, potentially turning hour-long jobs into five-minute tasks. Haven't implemented parallel processing for your LLM workflows yet? Watch the video I shared last month and grab the free workflow template. Your future self will thank you when that next big batch processing task lands on your desk.

This paper provides a detailed guide for healthcare professionals on implementing LLMs in medical workflows, focusing on best practices and methodologies to ensure effective, ethical, and safe deployment. 1️⃣ Key steps include task formulation, model selection, prompt engineering, fine-tuning, and deployment, each tailored to medical contexts for optimal LLM performance. 2️⃣ Tasks are divided into five categories: knowledge/reasoning, summarization, translation, structurization, and multi-modal data analysis, each requiring unique LLM capabilities and evaluation metrics. 3️⃣ Model selection should consider data sensitivity, context length, and compliance with standards like HIPAA, favoring HIPAA-compliant APIs or local models for patient data. 4️⃣ Prompt engineering techniques, including chain-of-thought and retrieval-augmented generation, improve response accuracy and explainability; fine-tuning is recommended when prompt adjustments are insufficient. 5️⃣ Deployment guidelines address regulatory compliance, equity, and continuous monitoring to minimize bias and ensure model reliability, with strategies to manage costs between proprietary and open-source models. ✍🏻 Qiao Jin, Nicholas Wan, Robert Leaman, Shubo Tian, Zhizheng Wang, Yifan Yang, Zifeng Wang, Guangzhi Xiong, Po-Ting Lai, Qingqing Zhu, Benjamin Hou, Maame A. Sarfo-Gyamfi, Gongbo Zhang, Aidan Gilson, Balu Bhasuran, Ph.D., Zhe He, PhD, FAMIA, Aidong Zhang, Jimeng Sun, Chunhua Weng, Ronald M. Summers, Qingyu Chen, Yifan Peng, Zhiyong Lu, PhD FACMI. Demystifying Large Language Models for Medicine: A Primer. arXiv. 2024. DOI: 10.48550/arXiv.2410.18856

Hopefully you've seen Jason Gorman lament that AI excitement around "speeding up coding" seems to miss that how fast the code is written is not a bottleneck we have. Bryan Finster consistently talks about having the basic skills in Continuous Delivery is required before AI can really help you deliver faster. However, I empathize that some of those topics require a lot of experience to understand what those actually mean. So let me break down the math for you; as an art student, I promise this will be simple. Pre AI: It takes me & my 15 co-workers an average of 2 to 5 days to get 1 line of code to production. With AI: It takes me & my 15 co-workers an average of 2 to 5 days to get 1 line of code to production. No change. We've replaced a Honda Civic "Hurry up & wait" w/ a Koenigsegg Regera. Being able go 249 mph instead of 124 mph doesn't help when stuck in traffic. Devs probably say LLM's help them code faster. Bottlenecks to the Lead Time (time in production - when you committed the code) are many, here are a few: - PR's / feature branches: these lead to delays in review, rework, merge problems which take lots of time. LLM's can refactor based on feedback, sometimes interpret a merge conflict w/o asking another dev why, but they can't "get your co-workers to stop what they're doing & review your PR right now". Even if you pair program with the LLM, currently, our LLM's aren't allowed to approve the PR, and we are waiting on 2 humans to approve. - Multiple Work in Progress: b/c of the above, devs will do multiple stories at once; now 2 things are delayed, but it "feels like we're doing a lot". While the LLM doesn't suffer a speed loss from context switching, devs do. - Rework: in the past 2 weeks, we've had to rework 2 API endpoints b/c we, all 19 of us, did not understand the business requirements & the API schema is incorrect. LLM's, even w/o hallucinations, access to our Google Docs (which have Gemini), "were prompted wrong" but technically were prompted "correct at the time". Google Bryan Finster's "The 3 wrongs". - Slow CI: It takes hours to get to Dev & QA (our unit & acceptance tests are fast, seconds & 1 minute). You'll find on Twitter & LinkedIn & YouTube tons of videos on AI for coding, maybe 2 people I know talk about AI in Observability, but despite the hate for YAML, I'm seeing no one talking about AI to speed up CI build times. Given it's a platform, I'd assume the platform team may possibly be saying things like "Copilot, please speed up this Groovy code & optimize our CloudFormation", but our PR's, even when approved, still take forever. These are a but a few of the many bottlenecks to Lead Time; e.g. "working software in user's hands". As you can hopefully see now, speeding up coding won't fix the 4 I cited. Using/learning techniques like Continuous Delivery, BDD, TDD, Team Topologies, Observability, & optimizing your DORA Metrics, you'll find a lot more value when using LLM's to "speed up coding".