Insights

Engineering Insights

Practical writing on software architecture, SaaS products, AI automation, legacy modernisation, and the business of building reliable systems.

RSS

Curated links from external sources — not 360Softy original articles.

ExternalSoftware Engineering

Google Developers Blog

Build Long-running AI agents that pause, resume, and never lose context with ADK

How to transition from stateless chatbots to production-grade agents capable of managing long-running enterprise workflows, such as HR onboarding, that span days or weeks. It introduces the Agent Development Kit (ADK) and its architectural shifts, specifically using durable state machines and persistent session storage to ensure an agent never loses context during "idle time" or server restarts. By leveraging event-driven webhooks and multi-agent delegation, the tutorial demonstrates how to buil

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

Supercharging LLM inference on Google TPUs: Achieving 3X speedups with diffusion-style speculative decoding

Researchers at UCSD have successfully implemented DFlash, a block-diffusion speculative decoding method, on Google TPUs to bypass the sequential bottlenecks of traditional autoregressive drafting. By "painting" entire blocks of candidate tokens in a single forward pass rather than predicting them one-by-one, the system achieved average speedups of 3.13x, with peak performance nearly doubling that of existing methods like EAGLE-3. This open-source integration into the vLLM ecosystem optimizes TPU

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

Building with Gemini Embedding 2: Agentic multimodal RAG and beyond

Google has announced the general availability of Gemini Embedding 2, a unified model that maps text, images, video, audio, and documents into a single semantic space. This model allows developers to process interleaved multimodal inputs in a single request, significantly improving performance for tasks like agentic RAG, visual search, and content moderation. By supporting over 100 languages and offering features like task-specific prefixes and Matryoshka dimensionality reduction, the model provi

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

Speeding Up AI: Bringing Google Colossus to PyTorch via GCSFS and Rapid Bucket

Google Cloud has introduced a high-performance integration that connects Rapid Storage directly to PyTorch via the fsspec interface to eliminate AI training bottlenecks. By utilizing Google’s Colossus architecture and bidirectional gRPC streaming, the solution offers up to 15 TiB/s aggregate throughput and significant reductions in latency. These improvements allow developers to speed up total training time by 23% with zero code changes required beyond updating the storage bucket type.

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

Building real-world on-device AI with LiteRT and NPU

LiteRT is a production-ready framework designed to help mobile developers unlock the power of Neural Processing Units (NPUs), overcoming the performance and battery limitations of traditional CPU or GPU processing. By providing a unified API that abstracts away hardware complexities, it allows industry leaders like Google Meet and Epic Games to deploy sophisticated AI models for real-time video, animation, and speech recognition with significantly higher efficiency. The platform further supports

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

Agents CLI in Agent Platform: create to production in one CLI

Google Cloud has introduced the Agents CLI, a specialized tool designed to bridge the gap between local development and production-grade AI agent deployment. The CLI provides coding assistants with machine-readable access to the full Google Cloud stack, reducing context overload and token waste during the scaffolding process. By streamlining evaluation, infrastructure provisioning, and deployment into a single programmatic backbone, the tool enables developers to move from initial concept to a l

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

Production-Ready AI Agents: 5 Lessons from Refactoring a Monolith

The blog post outlines the transition of a brittle sales research prototype into a robust production agent using Google’s Agent Development Kit (ADK). By replacing monolithic scripts with orchestrated sub-agents and structured Pydantic outputs, the developers eliminated silent failures and fragile parsing. Additionally, the post highlights the necessity of dynamic RAG pipelines and OpenTelemetry observability to ensure AI agents are scalable, cost-effective, and transparent in real-world applica

Google Developers BlogRead original

ExternalSoftware Engineering

Google Developers Blog

A2UI v0.9: The New Standard for Portable, Framework-Agnostic Generative UI

A2UI v0.9 introduces a framework-agnostic standard designed to help AI agents generate real-time, tailored UI widgets using a company’s existing design system. This update simplifies the developer experience with a new Agent SDK for Python, a shared web-core library, and official support for renderers like React, Flutter, and Angular. By decoupling UI intent from specific platforms, the release enables seamless, low-latency streaming of generative interfaces across web and mobile applications. I

Google Developers BlogRead original

Work with 360Softy

Building a SaaS product, AI system, or business platform?

Book a free consultation and we will tell you honestly whether we can help.

Book a consultation View our services →