A six-minute companion to thelong-form analysis. Read this if you have ten minutes and a calendar full of other meetings. Read the long-form if you have an hour and a strategy decision to make.

The 30-Second Version

Uber’s engineering team has published material describing a working internal platform that, by their own numbers, supports around sixty thousand weekly AI tasks flowing through a single governed gateway, with around ten thousand internal services exposed as agent-callable tools. The architectural pattern — a Model Context Protocol-centred fabric with a tool registry, central governance and auto-converted tool wrappers — is reproducible. The protocol is open.

Indian IT outsourcing — TCS, Infosys, Wipro, HCLTech, LTIMindtree, Cognizant, Tech Mahindra, Mphasis, Coforge — is the natural beneficiary of the same architecture applied to client estates rather than a single internal estate. The decisive question is not whether to build the platform. It is whether the firms that build it have the commercial nerve to convert the technical capability into a different deal shape. The technology is the easy part. The conversation about who gets paid is harder.

This brief covers the three things that matter most for an executive reading on a Tuesday: where the industry sits on the maturity curve, why the incentive conflict is the real constraint, and what the rollout looks like on a real account.

Where The Industry Sits On The Curve

Most outsourcers operate large client accounts at Stage 1 or Stage 2 today. RPA bots and per-tool copilots are useful, but they arepoint automation — each new tool or workflow improves one workflow. They do not compound.

Stage 3 is where compounding starts. An MCP-centred gateway with a tool registry, central governance, and auto-converted tool wrappers means every new tool exposed makes every existing agent more capable. The investment shape changes from “build a copilot” to “build a platform that produces copilots”. This is the published state inside Uber.

Stages 4 and 5 are where the commercial model shifts to outcome-billed delivery. Nobody is there at scale yet.

The honest gap from Stage 2 to Stage 3 is two to four quarters of platform work plus an unglamorous data-hygiene investment — CMDB cleanup, knowledge-base curation, ticket-taxonomy normalisation. The gap from Stage 3 to Stage 5 is not technical at all. It is the incentive problem.

The Incentive Problem Is The Real Constraint

This is the part of the thesis that gets understated in most conversations, including the original long-form draft of this analysis.

Outsourcing revenue is FTE-shaped. Contracts are priced as a stack of named roles multiplied by billing rates and headcounts. Reduce the headcount required to deliver a service and the immediate effect is a smaller bill to the customer. That is revenue cannibalisation, and it cuts against the platform business case.

The same effect applies inside the firm. A tower lead’s standing is shaped partly by tower size. A practice head’s position is shaped partly by billable headcount. The people who would have to champion the platform are also the people whose internal position it weakens. This is not a moral failing — it is rational behaviour given how the firm rewards them.

Four commercial paths through this, each failing in a different way.

Productivity dividend. Keep the FTE count, use the gains to expand scope or absorb new work without growing the team. Lowest internal resistance. Often produces no measurable customer benefit and therefore no commercial differentiation.

Margin retention. Hold the customer price, reduce delivery cost through automation, retain the margin. Works in shorter contracts. Creates trust issues at renewal — customers eventually see the AI talking points elsewhere and ask why they are paying yesterday’s price for tomorrow’s delivery model.

Outcome repricing. Move new contracts to per-ticket, per-incident or per-user pricing. The structural prize. Requires the commercial nerve to renegotiate large contracts, and the conversation is uncomfortable.

Productivity-billed sharing. Open-book pricing where automation gains are explicitly shared between vendor and customer. Mature, durable, hard to negotiate. Only viable in the small minority of high-trust accounts.

Customers also resist — not all, but a meaningful share — particularly in regulated industries, in accounts with hard data-sovereignty constraints, and where the customer’s own IT organisation has a vested interest in keeping the outsourcer’s headcount visible.

The firms that win will build the platformand have the commercial conversation. The firms that build the platform without the conversation will end up with a quietly impressive internal capability that does not show up in revenue.

What This Looks Like For A Real Account

A composite client. Eighty thousand endpoints across thirty countries. Around four hundred and fifty named FTEs across all towers (a typical 1:175 endpoint-to-FTE ratio). Around ninety-five thousand monthly tickets, of which roughly forty-two thousand are repetitive L1 categories. Five-year contract with a single Indian outsourcer. Moderate CMDB hygiene. Well-maintained ServiceNow; inconsistent integration with endpoint management.

Phase 1 (months 0–3). MCP gateway stood up. Three tools exposed: ServiceNow, Active Directory, Intune. One agent built: L1 password-reset and account-unlock automation. No-code builder wired. Baseline measured. Six gateway calls per minute by week ten.

Phase 2 (months 3–9). Orchestrator agent and VDI-triage workflow live. SCCM, Splunk, vSphere, NetBackup exposed. Fifteen SDMs and twenty senior engineers trained on the builder. L1 auto-resolution stabilises around 22 percent. Headcount has not changed yet, but queue depth has dropped by around 30 percent and customer satisfaction has lifted by single-digit points.

Phase 3 (months 9–18). Onboarding workflow live. Emergency patch workflow live. Asset and license reconciliation agents live. L1 auto-resolution stabilises around 32 percent. The customer asks why the monthly report has improved so visibly. The renewal conversation opens.

Phase 4 (months 18–24). Renewal lands on a hybrid commercial model — partial FTE billing plus a per-ticket envelope. Around four percent of the original FTE base is redeployed to other accounts. Another eight percent shifts to higher-value advisory work. The remainder continues in delivery roles. The vendor’s blended margin on the account improves by approximately two to four percentage points despite a lower headline contract value. Customer-side, cost-per-incident reduces by around 18 to 22 percent.

The metrics that actually decide success are not the headline ticket numbers. They are therenewal terms and theredeployment ratio. The renewal tells you whether the platform investment converted into a defensible commercial position. The redeployment ratio tells you whether the firm managed the internal political conversation well enough to keep its experienced people inside the business.

The bank is fictional. The numbers should be read as illustrative.

What To Do This Quarter

Three concrete actions for the executive reading this with a calendar full of other meetings.

One. Pick a single client account and a single process. Stand up an MCP-centred gateway against it. Two-quarter timeline. Real money — somewhere between $500K and $2M depending on starting position and tool count. Password reset and account unlock are the canonical starting workflows because the volume is enormous and the failure modes are well-understood.

Two. Open the commercial conversation in parallel with the technical build. Do not wait for delivery. Renewal cycles are long, and the conversation about deal shape needs months of customer education before it can convert. The technical platform and the commercial repositioning are two sides of the same investment.

Three. Pick the right SDM and the right tower lead to champion the rollout. The wrong choice will produce technical success and commercial failure. The internal politics of automation are as important as the architecture — possibly more.

Where To Go Deeper

The full analysis — protocol-versus-platform distinction, competitive landscape, role taxonomy, three concrete workflows in detail, the risks landscape, the five-phase rollout path — is in the long-form companion piece:From Service Desk to Agent Fabric.

The work that distinguishes the firms that win from the firms that wait is the execution discipline applied to both halves of the problem. The platform is the easier half. The conversation about who gets paid for the productivity is the harder half. Both must happen at the same time.

Uber’s engineering team has published material describing a meaningful shift in how they operate AI inside the company. They have moved Model Context Protocol (MCP) from a pilot into a standardised internal platform. Their published numbers — in the engineering blog and the accompanying conference talk linked at the foot of this post — describe in the order of sixty thousand weekly AI tasks flowing through a centralised gateway, with around ten thousand internal services exposed as agent-callable tools. Engineers and non-technical staff compose those tools into workflows through either a no-code builder or SDKs, with governance, redaction and access control inside the platform rather than in each agent.

That is interesting on its own technical merits. It becomes much more interesting if you sit in Bangalore, Pune, Hyderabad, Chennai or Noida and run a large infrastructure-delivery organisation for global clients. The pattern Uber has industrialised internally is, in shape, the same pattern that Indian IT outsourcing has been circling around for years without quite landing.

This article walks through what Uber appears to have built, separates that confirmed picture from my own extrapolation into the outsourcing world, places it in a maturity model, and addresses the political and commercial reasons why the rollout is harder than the architecture diagrams suggest.

If you only have six minutes, there is acompanion executive briefing covering the maturity model, the incentive conflict and the bank case study. This is the long form.

What’s Verified, What’s Mine

Before going further: what Uber has publicly described — the gateway pattern, the auto-conversion of service endpoints into MCP tools, the no-code builder, the weekly-task and service-count numbers — is treated here as theconfirmed picture, citable to their public material. The specific figures (60,000 weekly tasks, 10,000 services) come from Uber and should be verified against the linked sources rather than taken on my authority.

Everything that follows about Indian IT outsourcing — the role taxonomy, the workflow examples, the economic envelope, the rollout phases, the incentive analysis, the bank case study — ismy extrapolation, informed by fifteen years inside the industry. It is not an Uber statement and should not be read as one.

That separation matters because the Uber numbers travel well — they get repeated in pitch decks and LinkedIn posts without the surrounding caveats — and outsourcing-specific claims get attributed back to Uber in ways that misrepresent what they actually said.

Protocol Versus Platform

A precision point worth getting right up front, because technical readers will object if it is left fuzzy.

MCP itself is a protocol. It specifies how an AI agent describes tools, calls them, and consumes their results. It is open, broadly adopted, and useful. The protocol on its own does not build a platform. HTTP is a protocol — the World Wide Web is the vast set of conventions, infrastructure and tooling built on top of it. MCP sits in the same relationship to a platform like the one Uber has built.

What Uber has built is not “MCP”. It is agateway, registry, governance and builder platform that uses MCP as its interface contract. The protocol is the load-bearing standard. The platform around it is where most of the engineering investment sits.

For the rest of this article, when I say “the Uber MCP stack” or “an MCP-centred fabric”, I mean the protocol-and-platform combination. When I mean the protocol itself, I will say “the MCP specification”.

Uber’s Build, In Four Pieces

Before standardising MCP, Uber’s AI initiatives looked like most large engineering organisations’ — multiple teams, multiple stacks, multiple ways of calling internal services, and every new agent requiring hand-wired integrations against an estate of around ten thousand services. Governance was worse: each integration had its own access pattern and (often non-existent) data-redaction story. Uber has been candid in their public material that this was the state they were trying to change.

What they built has four moving parts.

Agateway sits between every agent and every internal service. AI traffic terminates there. Authentication, authorisation, rate-limiting, logging and PII redaction happen at a single layer, so that sensitive-data handling is done once rather than once per agent.

Aregistry catalogues the tools an agent can call. The decisive design choice — and the source of most of the engineering value — is that Uber did not require every team to hand-write an MCP tool definition. Their toolingauto-converts existing service endpoints into MCP tools. New service ships, new tool appears, agents pick it up.

Ascanning and access-control layer runs inside the gateway. Every tool registered goes through risk scanning. Every call is bound to a named identity with scoped permissions and is logged.

Abuilder layer sits on top: a no-code workflow composer for non-engineers, and SDKs for engineers. Both produce workflows that run through the same gateway against the same registry. There is no second-class tier of tools or governance.

Uber’s material also describes a future direction — askills repository that curates and shares verified multi-step workflows across the organisation, on the model of an internal app store.

What’s Actually Novel

Most of the individual components are not new. API gateways are decades old. Tool registries exist in many forms. Governance is well-trodden territory. What is distinctive is thecombination at scale, with one open protocol as the contract, and with auto-generation of tools from existing service definitions.

Specifically: the auto-conversion of endpoints into MCP tools is the multiplier. Without it, exposing ten thousand services as agent-callable would be a years-long manual programme, and most organisations stop at a few hundred. With it, the marginal cost of bringing a new service into the agent surface area drops close to zero, and the platform compounds.Every new tool added makes every existing agent more capable. That compounding is the architectural insight most observers miss.

The single governance chokepoint is the second distinguishing characteristic. Many enterprises have built point AI integrations; few have built one place through which every AI call passes, logged, audited and scoped. That single chokepoint is what makes the platform credible to a public-company audit function.

What It Bought Them

The published numbers — around sixty thousand weekly AI tasks and ten thousand services exposed — are the headline. The qualitative outcomes Uber describes matter as much: governance moving from per-project negotiation to platform property, every new tool exposed making every existing agent more capable, and the population of people able to compose workflows expanding well beyond the AI engineering team.

Verify the methodology against the linked engineering blog and conference talk before quoting these figures further. The definitions of “task” and “service” matter, and pitch-deck repetition tends to lose the qualifiers.

Why MCP And Not An Existing Platform

A reasonable question: why would an outsourcer build an MCP-centred fabric rather than adopt one of the existing platforms already in this space?

ServiceNow’s AI agents and Workflow Studio are strong when ServiceNow is the system of record, weaker when the client estate is multi-vendor.

Microsoft Copilot Studio is excellent if the estate is heavily M365/Azure-centric. The trade-off is platform lock-in and a relatively closed extensibility model.

Moveworks and Aisera are specialist L1-automation vendors — strong out-of-the-box for tickets and natural-language ITSM, less flexible beyond the service desk.

PagerDuty AIOps targets incident response and on-call; not designed as an estate-wide fabric.

UiPath, Automation Anywhere and Blue Prism are RPA platforms with AI features layered in; they work, particularly for legacy systems without APIs, but were not designed around the agent-orchestration model.

Atlassian Rovo, Cortex, and OpenAI operator-style systems are credible newer entrants, worth tracking particularly when the underlying platform is already in use.

The case for an MCP-centred fabric is specifically protocol-openness and multi-tenant flexibility. An Indian outsourcer running infrastructure for thirty different clients does not want to bet on each client’s chosen vendor stack. They want a fabric that can wrap any client’s tool surface — ServiceNow at client A, BMC at client B, Cherwell at client C — into the same agent-callable shape. MCP is the only protocol-level standard that makes that portable.

The honest counterpoint: the MCP ecosystem is younger than any of the platforms above, and the per-tool engineering effort to build wrappers is real. The right answer for most outsourcers is probablyboth — use Copilot Studio or ServiceNow AI inside specific engagements where platform fit is strong, and stand up an MCP-centred fabric as the cross-client integration substrate that owns multi-tenant governance.

The Outsourcing Mirror

Replace “Uber’s ten thousand internal services” with“a client’s BigFix, SCCM, Intune, ServiceNow, Active Directory, Azure, vSphere, NetBackup, Splunk, CrowdStrike, Citrix DaaS and forty other tools” and the structural picture is the same. Indian IT outsourcing companies run the infrastructure for a large share of the Global 2000. The same constellation of vendor tools, the same heavy ITIL process layer, the same delivery pyramid of engineers operating that process at scale.

A typical infrastructure engagement touches forty to one hundred discrete tools, most with an API and none currently agent-safe. ITIL workflows are structured, repeatable and multi-system — the exact shape an orchestration platform is designed to industrialise. A large outsourcing tower processes hundreds of thousands of tickets a month, of which a meaningful proportion is highly repetitive.

The genuinely difficult difference is multi-tenancy. Uber operates one estate. An outsourcer operates many. The platform has to be multi-tenant by design — strong client isolation, separate audit boundaries, per-client data sovereignty. That is materially harder than the Uber single-tenant build.

The Service Desk Becomes A Routing Layer

A working taxonomy of roles inside a typical infrastructure outsourcing tower, sorted by how aggressively the MCP-centred pattern can be applied.

Fully automated covers password resets, account unlocks, drive and printer mappings, mailbox quota requests, software-catalogue requests, license assignment, standard patch deployment, backup-job verification, daily dashboards, and joiner-leaver provisioning. Structured inputs, well-defined steps, known failure modes. Agents run them; humans review the exception queue.

Agent-led with human approval covers production change execution, privileged access grants, firewall rule additions, DNS changes — anything that would have gone through a CAB. The agent drafts the change record, attaches the technical justification, runs pre-change validation, and produces the back-out plan. A human approves with a click.

Human-led with agent assist covers L2 and L3 incident analysis, root cause analysis, architecture review, capacity planning, vendor escalation, major-incident communications and service-review preparation. The agent gathers diagnostics, summarises logs, retrieves relevant knowledge-base material, and drafts a first version. The human applies judgment, two to four times faster.

Human-only, at least for now, covers Sev-1 crisis leadership with customer C-suite involvement, contract negotiation, performance management, and the conversations that require reading what is not being said. The agent can prepare the brief; the conversation is yours.

The role that changes most is theL1 engineer. Most of the L1 queue today is repetitive triage; after MCP, most of that volume is auto-resolved before reaching a human. The L1 engineer ends up doing what L2 does today — the genuinely novel cases. The role population shrinks but the people in it operate at a higher level.

Thepatch-management engineer moves from running deployments to owning strategy. Theasset and license manager shifts from reconciliation to vendor negotiation. TheSDM stops stitching reports together and starts having strategic conversations.L3 specialists are the least affected in volume but the most augmented in capability — the same person designing a thirty-thousand-endpoint patching strategy, with a domain agent at their side that knows every Fixlet and every advisory.

Three Workflows In Detail

Three concrete workflows, each composed from agents calling registered tools through the gateway. Every system referenced is one Indian outsourcers already run for clients.

A — VDI session won’t connect

Today, a ticket of this shape sits in the L1 queue for fifteen to forty-five minutes before an engineer picks it up, asks for basic information the user has not provided, checks four or five systems by hand, and either resolves it or escalates.

With an MCP-centred fabric, the ticket arrives at an orchestrator agent that recognises the symptom class and dispatches in parallel — an Active Directory agent checks account status and recent authentication failures; an Intune agent checks device compliance and last-seen timestamp; a Citrix agent checks session-host availability and active maintenance windows. Within seconds the orchestrator has a structured diagnostic.

If the issue is a known pattern, the orchestrator resolves directly or routes the user to the right self-service path. If the diagnostic shows something unusual, the orchestrator escalates to L2 with the full brief pre-attached. The L2 engineer reads a one-paragraph summary and makes a decision, rather than starting from zero in four different tools.

B — New joiner onboarding

HR fires a “new hire confirmed” event. An onboarding orchestrator reads the role profile and location. An identity agent creates the AD account and provisions the M365 mailbox. A licensing agent assigns the right SKUs. An endpoint agent triggers the laptop build. A collaboration agent adds the new joiner to relevant Teams and SharePoint sites. A VPN agent provisions remote access. A notification agent sends the welcome packet and notifies the manager.

Every step is logged. Every approval that requires a human — sensitive group memberships, elevated privileges, anything outside policy — is routed as a single one-click decision. What used to be a three-to-five-day, twelve-touchpoint process across three teams becomes a four-to-eight-hour automated workflow with a clean audit trail.

C — Emergency patch deployment

A vendor publishes a critical advisory. A vulnerability agent parses it. A CMDB agent identifies affected assets. A risk agent calculates the blast radius. A change agent drafts the emergency CAB record — justification, test plan, back-out plan, customer notification — and routes for one-click approval. A patch agent executes the pilot deployment. A monitoring agent watches for regressions. If no regressions, broader rollout proceeds in stages. A reporting agent generates the customer report at each stage.

The patch-management engineer’s role here is to set the strategy, define the pilot group, review the monitoring envelope, and approve the rollout cadence. One engineer, around an engineer-hour in the loop and four-to-eight hours of wall-clock execution, on a workflow that previously consumed two to three engineers across one-and-a-half days.

From FTE Arbitrage To Outcome Arbitrage

The strategic case is more persuasive with numbers. The figures below aredirectional, my own estimates based on what I have seen in infrastructure delivery — not Uber numbers and not benchmarks. Real client engagements will move these by a factor of two or three either way depending on tower maturity, tool quality and data hygiene.

These are platform-success-case figures, not industry averages. Realistic deployments will hit the lower bound of the “after” column unless three conditions are met: clean CMDB hygiene, well-classified ticket taxonomy, and an MCP-centred fabric covering at least 60 percent of L1 ticket categories. The “before” column reflects mid-range offshore-billed contracts; US- and Europe-billed contracts run two to three times higher on the L1 cost line.

Metric	Today (typical)	With MCP-centred fabric
Password reset, end-to-end	8–15 min	1–2 min
L1 ticket auto-resolution rate	5–15%	25–40%
MTTR, repetitive L1 incident	30–60 min	2–5 min
MTTR, L2 incident	90–240 min	60–180 min
New-joiner onboarding cycle	3–5 business days	4–12 hours
Emergency patch deployment	2–3 engineer-days	1 engineer-hour (4–8 hr wall-clock)
L1 ticket cost (fully-burdened, offshore)	$5–$25	$0.50–$1.50
Change-record drafting	30–60 min	5–10 min
SDM reporting overhead (per week)	6–10 hours	2–3 hours
License optimisation cycle	quarterly manual	continuous

The L1 auto-resolution rate is the line worth interrogating most carefully. Public Moveworks and Aisera figures advertise 40–60 percent on flagship deployments, but the deployed-average across their customer base is closer to 25–35 percent. The 25–40 percent range above reflects that deployed-average reality, not the marketing number.

The commercial implication, if these numbers hold even directionally, is the structural shift from billed-FTE to billed-outcome. That shift has been talked about in this industry for a decade. The MCP-centred pattern is the first thing I have seen that makes it plausibly executable. Whether the industry actually executes it is a different question — and the most interesting one in this whole article.

The Incentive Problem

The hardest problem here is not technical. It is commercial and political, and the previous version of this article underplayed it.

Outsourcing revenue is FTE-shaped. A typical infrastructure deal is priced as a stack of named roles — L1 engineers, L2 engineers, L3 specialists, SDMs, project managers, change managers — multiplied by a billing rate and a headcount, with volume commitments and tiered SLAs layered on top. When an outsourcer reduces the headcount required to deliver a given service, the immediate effect is a smaller bill to the customer. That is revenue cannibalisation, and it cuts directly against the platform business case.

The same is true at the individual delivery-team level. A tower lead’s annual review is partly shaped by the size of the tower. A practice head’s standing inside the firm is partly shaped by the practice’s billable headcount. When automation reduces both, the people who would have to champion the platform are also the people whose internal position it weakens. This is not a moral failing on their part — it is rational behaviour given how the firm rewards them.

There are four paths through this, and they tend to fail in different ways.

Path one — productivity dividend. Keep the FTE count and use the productivity gains to expand scope or accept new work without growing the team. This is the path of least internal resistance but the one most likely to produce no measurable customer benefit and therefore no commercial differentiation.

Path two — margin retention. Hold the customer price flat, reduce the delivery cost through automation, retain the margin. This works in shorter contracts but creates trust issues at renewal — customers eventually see the AI talking points elsewhere and ask why they are paying yesterday’s price for tomorrow’s delivery model.

Path three — outcome repricing. Move new contracts to per-ticket, per-incident or per-user pricing where automation directly drives margin. This is the structural prize but requires commercial nerve, and the renegotiation conversation is uncomfortable. Existing customers will ask why their renewal terms are changing; new customers will ask whether the model has been proven elsewhere first.

Path four — productivity-billed sharing. Open-book pricing where automation gains are explicitly shared between vendor and customer. Mature, durable, hard to negotiate. Requires the kind of trust relationship that exists in a small minority of outsourcing accounts.

Customers will also resist. Not all customers, but a meaningful share — particularly in regulated industries where automated change execution is uncomfortable, where data-sovereignty constraints make agentic processing politically difficult, or where the customer’s own IT organisation has a vested interest in keeping the outsourcer’s headcount visible and known.

None of this is a reason not to build the platform. It is a reason to plan the commercial conversation as carefully as the technical one. The firms that succeed will be the ones that build the platformand have the commercial discipline to convert the technical capability into a different deal shape. The firms that build the platform but cannot have the conversation will end up with a quietly impressive internal capability that does not show up in revenue.

Where This Can Go Wrong

The first version of this article underplayed operational complexity. The honest list of failure modes.

Automation blast radius. An agent that can change firewall rules across an estate can break that estate at machine speed. Mitigations: layered approvals, staged rollouts, kill switches, per-action blast-radius modelling.

Incorrect remediations. The agent confidently executes a fix that is wrong for the specific environment. Mitigations: bias toward propose-not-execute for production actions, confidence scoring on every action, source attribution.

Data quality dependency. The platform is only as good as the data it relies on. The CMDB problem is the most visible — agents need accurate asset inventory for identification, blast-radius calculation and change correlation, and most long-running outsourcing engagements have CMDB hygiene problems that have been tolerated for years. Beyond CMDB, the same applies toknowledge-base quality (agents grounded in stale or contradictory KB articles produce confidently wrong answers),ticket-taxonomy quality (poorly classified incident categories destroy the orchestrator’s routing accuracy),inconsistent naming and tagging (the same server appearing as three different identities across tools), and thetribal-knowledge problem (the senior engineer who knows the one production server that must not be patched on Wednesdays because of a quirk that was never documented). Standing up the fabric forces these issues into daylight, which is uncomfortable in the short term and necessary in the long term.

Privilege escalation. Scopes too broad means agent compromise gives lateral movement. Mitigations: minimum-privilege per workflow, separate identities per agent, secret rotation, prompt-injection defences.

Audit, legal and data sovereignty. Cross-border inference is a real audit issue; contracts often constrain where prompts can travel. Mitigations: region-pinned gateway, sovereign model selection, redaction before any cross-border hop.

AI-induced outage. Tool-wrapper regression, model behaviour change, or workflow loop can cascade. Mitigations: canary tool deploys, per-client model pinning, workflow tests in CI, clear rollback story.

Over-automation. Removing humans from too many steps removes the implicit knowledge transfer that keeps L2 and L3 engineers sharp. The L1-to-L2 promotion path was always partly about pattern recognition built up over thousands of tickets. Mitigations: rotate humans through agent-handled work for training, design agent output as a learning surface rather than a black box.

Integration brittleness. Auto-generated tool wrappers are not always semantically correct — the wrapper might technically work but get the underlying business logic subtly wrong. Mitigations: human review of high-stakes wrappers, regression test suites at the tool level, a tiered trust model (draft → verified → skills-grade).

None of these is a reason not to build the platform. All are reasons to build it with the operational discipline that distinguishes a working enterprise platform from a clever demo.

A Maturity Model

The pattern fits into a broader six-stage model that locates outsourcers against where the industry is heading.

Stage	Capability	Where it sits
0	Scripts (PowerShell, Bash, ad-hoc)	Legacy operations
1	RPA (UiPath, Blue Prism — UI-driven)	Most outsourcers today
2	Copilots (Moveworks, Aisera, Copilot Studio — per-tool LLM helpers)	A growing share of leading-edge engagements
3	MCP gateway (single chokepoint, registry, governed tools)	Uber’s published state
4	Multi-agent workflows (orchestrators, specialists, skills repository)	Very few in production at scale
5	Outcome-billed delivery (per-ticket, per-incident, per-user pricing)	None at scale yet

The ceiling between Stage 2 and Stage 3 is where the platform investment matters. Everything before Stage 3 is point automation — useful, productive, but not compounding. Stage 3 is where every new tool exposed makes every existing agent more capable, and where governance becomes a property of the platform rather than the agent. Stages 4 and 5 are where the commercial model shifts.

Most outsourcers reading this article sit at Stage 1 or 2 for their largest accounts. The honest gap to Stage 3 is two to four quarters of platform work plus the unglamorous data-hygiene investment described in the risks section. The gap from 3 to 5 is the incentive problem above.

Case Study: An 80,000-Endpoint Bank

To anchor the abstractions, a fictional but representative client journey.

A global bank with 80,000 endpoints across thirty countries, currently outsourced to a single Indian IT provider on a five-year contract. The current delivery footprint is roughly 450 named FTEs across all towers (service desk, endpoint, identity, cloud, network, security operations) — an endpoint-to-FTE ratio of around 1:175, which is typical for a well-run but not bleeding-edge tower. Monthly ticket volume sits around 95,000, of which roughly 42,000 are repetitive L1 categories. The customer’s CMDB hygiene is moderate; their ITSM (ServiceNow) is well-maintained but the integration with their endpoint management estate (a mix of SCCM and Intune) is inconsistent.

Phase 1 (months 0–3). Stand up the MCP gateway. Expose three tools: ServiceNow, Active Directory, Intune. Build one agent: L1 password-reset and account-unlock automation. Wire the no-code builder. Measure baseline. Six gateway calls per minute by week ten.

Phase 2 (months 3–9). Expose SCCM, Splunk, vSphere, NetBackup. Build the orchestrator agent and the VDI-triage workflow. Train fifteen SDMs and twenty senior engineers on the builder. By month nine, around 22 percent of L1 volume auto-resolves; the L1 headcount has not changed yet, but the queue depth has dropped by around 30 percent and customer satisfaction has lifted by single-digit points.

Phase 3 (months 9–18). New-joiner onboarding workflow goes live. Emergency patch workflow goes live. Asset and license reconciliation agents go live. The L1 auto-resolution rate stabilises around 32 percent. The bank’s customer asks why the SDM monthly report has improved so visibly. The renewal conversation begins.

Phase 4 (months 18–24). Renewal lands on a hybrid model — partial FTE billing plus a per-ticket envelope. Around four percent of the original FTE base is redeployed to other accounts; another eight percent shifts to higher-value advisory work; the remainder continues in delivery roles. The vendor’s blended margin on the account improves by approximately 2–4 percentage points despite the lower headline contract value. Customer-side, the cost-per-incident reduces by around 18–22 percent.

The interesting metrics in this case are not the headline ticket numbers. They are the renewal terms and the redeployment ratio. The renewal is what tells you whether the platform investment converted into a defensible commercial position. The redeployment ratio is what tells you whether the firm managed the internal political conversation well enough to keep its experienced people inside the business.

The fictional bank is a composite, not a real client, and the numbers should be read as illustrative rather than predictive.

The Rollout Path

A staged path for an outsourcer starting today. The phases are deliberately conservative.

Phase 1 (months 0–3). Pick one client, one process. Stand up a per-client MCP gateway. Expose three to five tools. Build governance from day one: scanning, redaction, audit logging, scoped access, secret rotation. Build one specialist agent. Measure baseline before anything is turned on.

Phase 2 (months 3–6). Build the no-code builder layer. Train ten SDMs and ten senior engineers. Compose the first three workflows — password reset, account unlock, basic VDI triage. Publish the numbers internally.

Phase 3 (months 6–12). Expose the rest of the client’s estate. Build specialist agents per major tool. Compose the next ten workflows. Begin the internal skills-repository pattern, curated rather than free-for-all.

Phase 4 (year 2). Replicate the platform pattern to the next two or three clients. Each new client engagement still requires substantial per-client work — tenant isolation, data sovereignty, CMDB onboarding, integration with each client’s specific tool versions, contract renegotiation, change management with the client’s IT organisation. Implementation time improves from quarters to a few months, not weeks.

Phase 5 (year 3+). Outcome-billed contracts begin to make sense with two or three reference clients in production. The commercial nerve to renegotiate large multi-year contracts on a new basis is as important as the technology by this point.

The substantial work that remains, even after the platform pattern is established, includes identity federation across client tenants, robust RBAC for multi-tenant agent access, rollback safety for agent-executed changes, hallucination containment in production conditions, change-governance integration with each client’s existing CAB, observability across the full stack, and workflow determinism for processes with regulatory SLAs. These are the issues that determine whether the platform delivers in production or merely demos well.

The Third Wave

Indian IT outsourcing has been through three waves. Offshoring in the 1990s and early 2000s — moving work from where it was expensive to where it was cheap. Digital transformation in the 2010s — moving customers from legacy stacks to cloud, mobile and modern collaboration. The third wave is automation-led services — moving the unit of delivery from the FTE to the agent-augmented workflow.

The companies that build a credible MCP-centred fabric first will set the terms for the decade that follows. The ones that wait will negotiate with customers who have done the math and know what the cost-per-ticket should be in a world where an agent handles a meaningful share of the volume.

Uber’s published material is the most usable public case study to date of what such a platform looks like in production at scale. The architecture is reproducible. The protocol is open. The operational discipline required is well within what mature outsourcing organisations already have. The incentive conversation is harder than the engineering, and it is the part that decides who wins.

The technology has been ready for two years. The platform pattern is now visible. The work that remains is execution, and the conversation about who gets paid for it.

Sources & Notes

Uber claims in this article reference the company’s publicly available engineering material on MCP:

• The conference talk:https://youtu.be/yVqMxBahjfA
• Uber’s engineering blog athttps://www.uber.com/blog/engineering/ — search for their MCP and AI platform posts.
• The Model Context Protocol specification athttps://modelcontextprotocol.io

Vendor capabilities in the competitive-landscape section reflect publicly available product documentation as of 2026 and move quickly; verify with vendors directly before procurement decisions.

Economic envelope figures, the role taxonomy, the workflow examples, the maturity model, the bank case study and the incentive analysis are my extrapolation based on fifteen years in infrastructure delivery. Not benchmarked, not audited, not validated by any of the firms named. Treat as a starting hypothesis for a client-specific business case, not as a benchmark.

Where the article uses “could”, “would” or “should”, that is deliberate hedging. The MCP-centred pattern is a working architecture inside Uber; whether it works equivalently inside a multi-tenant outsourcing environment is a thesis I find credible but cannot yet point to a published reference for.

Feedback, corrections and counter-evidence are welcomed.

This article documents the design and implementation of a domain-specific Retrieval-Augmented Generation (RAG) conversational assistant. The project was completed as theWeek 15 Graded Mini Project of theIITM Pravartak Professional Certificate Programme in Agentic AI and Applications. The brief required the construction of an assistant that retrieves content from a private document set and produces answers grounded in that retrieved context, while refusing to answer when supporting evidence is absent.

The implementation selects theTechnology domain and indexes six official pages of the GitHub REST API documentation. The assistant accepts free-form questions in a command-line interface, performs context-aware follow-up handling, returns concise answers with source attribution, and emits a fixed refusal phrase when the retrieved context does not contain the answer.

Architecture

The pipeline is split into two phases. An offline ingestion stage prepares a reusable vector index; a runtime stage answers user turns by retrieving from that index and prompting a large language model under strict grounding instructions.

Public Data Sources

Six public pages from the official GitHub Docs were saved locally as plain-text files indata/raw/:

• Quickstart for GitHub REST API
• Getting started with the REST API
• Authenticating to the REST API
• Rate limits for the REST API
• Best practices for using the REST API
• Troubleshooting the REST API

These pages were chosen because they form a self-contained operational knowledge set: a developer integrating with the API typically needs to consult exactly this material when handling authentication, rate-limit pressure, and recovery from common errors.

Components and Concepts

The following components and ideas underpin the implementation. Each is summarised briefly to make the design choices easy to evaluate.

Retrieval-Augmented Generation (RAG). A pattern in which a language model is grounded on retrieved passages from an external corpus rather than relying solely on parametric memory. Retrieval reduces hallucination and lets the assistant cite source material it actually consulted.

LangChain. The orchestration framework used throughout the project. It provides composable abstractions for document loaders, text splitters, embeddings, vector stores and chat models, which keeps the ingestion and runtime code small and idiomatic.

RAG Chunking. Long documents are split into smaller passages before embedding. This implementation usesRecursiveCharacterTextSplitter with a 900-character chunk size and a 150-character overlap. Splitting prefers semantic boundaries (paragraph breaks and Markdown headings) so that each chunk is internally coherent and retrieval surfaces meaningful units rather than fragments.

OpenAI Embeddings. Each chunk is converted into a high-dimensional vector using thetext-embedding-3-small model. The vector captures the semantic content of the chunk so that questions phrased differently from the source text still retrieve the right passages.

FAISS. Facebook AI Similarity Search is the local vector store used to index the embeddings. The index is persisted to disk undervectorstore/github_rest_api_faiss so that ingestion runs once and the runtime simply loads the existing index, satisfying the rubric’s reusable-index criterion.

Top-k Retrieval. At each user turn the retriever returns thek=4 chunks most similar to the query. A smallk keeps the prompt focused and reduces the chance that off-topic passages dilute the grounded answer.

Follow-up Rewriter. A short, dedicated LLM call rewrites the user’s latest question into a standalone retrieval query using recent conversation history. This is the mechanism that allows ambiguous follow-ups such as “what about rate-limit headers?” to be embedded and retrieved correctly without the model having to guess the missing antecedent.

Grounded Answerer. The final answer is produced bygpt-4o-mini at temperature zero under a system prompt that instructs the model to answer only from the retrieved context, append aSources: line naming the document titles consulted, and otherwise return the exact refusal phrase.

Refusal Behaviour. When retrieval returns no usable context, the runtime short-circuits and emits the required string verbatim:I don't have enough information in the provided documents. The refusal is also enforced by the system prompt, providing defence in depth against hallucinated answers.

Implementation Layout

The submission is organised as two thin Python modules and a Jupyter notebook.ingest.py loads the text files, removes GitHub Docs templating syntax, extracts a human-readable title from each document, chunks the text, embeds the chunks, and writes the FAISS index.chatbot.py loads the persisted index, manages conversation history, invokes the rewriter and the grounded answerer in sequence, and prints the response with source attribution. The accompanying notebook imports the same functions so that the notebook and the CLI execute identical code paths, eliminating drift between the two surfaces.

Configuration is environment-driven. The OpenAI API key is read from a.env file and an optionalOPENAI_BASE_URL is honoured so that the same code runs against the Vocareum-provided proxy without modification. No secrets are committed to source.

Sample Conversation

A short transcript illustrates the three behaviours the rubric tests — grounded answer, follow-up resolution, and refusal.

You: What headers should I send when making requests to the GitHub REST API?Assistant: GitHub REST API requests commonly use an Accept header such asapplication/vnd.github+json. Authenticated requests send an Authorizationheader with Bearer YOUR-TOKEN. GitHub also recommends a valid User-Agentvalue, such as your GitHub username or application name.Sources: Getting started with the REST API; Authenticating to the REST APIYou: What about rate limit headers?Search query: rate limit response headers GitHub REST APIAssistant: Response headers can include x-ratelimit-remaining andx-ratelimit-reset, which indicate how many requests remain in the currentperiod and when the rate limit resets.Sources: Getting started with the REST API; Rate limits for the REST APIYou: How should an integration handle rate limit errors?Assistant: If a retry-after header is present, wait that many seconds beforeretrying. If x-ratelimit-remaining is 0, wait until the time given byx-ratelimit-reset. Otherwise, wait at least one minute and then useexponentially increasing waits for repeated secondary rate-limit failures.Continuing to make requests while rate-limited may result in a ban.Sources: Best practices for using the REST API; Troubleshooting the REST API

The second exchange demonstrates the follow-up rewriter at work: the original “what about rate limit headers?” is expanded into a self-contained search query before retrieval, which is what allows the retriever to surface the correct passages despite the missing antecedent.

Now, a deliberately off-topic question — one the index cannot possibly support — triggers the safety path:

No chunk in the index supports an opinion on hardware, so the assistant returns the mandated refusal verbatim rather than producing a plausible-sounding but ungrounded answer. The refusal is enforced twice — once procedurally when retrieval returns empty, and once in the system prompt — so the behaviour is stable even when retrieval surfaces weakly related chunks.

Closing Observations

Three observations stand out from this exercise.

• Retrieval quality is set upstream, not by the vector store. The quality of retrieval is determined less by the choice of vector store than by the upstream decisions about cleaning and chunking. Stripping GitHub Docs templating syntax — the{% data ... %} Liquid blocks, the reusables, and the[AUTOTITLE] link macros — materially improved the relevance of returned chunks because the embeddings stopped clustering around boilerplate.
• Follow-up handling must be engineered, not assumed. It is not a free behaviour of the language model. A separate rewriter step that converts conversational queries into standalone search queries is the smallest reliable mechanism, and it removes a category of retrieval failures that would otherwise be invisible in casual testing.
• Refusal is a feature, not a fallback. Enforcing the refusal both procedurally (short-circuit on empty context) and in the system prompt is what gives the assistant a useful posture in production: it will say nothing it cannot support, and it will say so in a predictable way.

Ask a general-purpose LLM how to deploy an emergency Microsoft patch to ten thousand endpoints through HCL BigFix Workspace+, and you will get a plausible answer. Read it carefully and you will also find quietly invented menu paths, mis-named Fixlet sites, and “best practices” that no BigFix administrator has ever followed. The model has clearly seen the product once or twice during pre-training, just not enough to be useful when the deployment window opens at 10pm on a Tuesday.

This is the gap that specialist agents are meant to close. Not by retraining a foundation model — that is expensive, slow, and almost never the right tool — but bygrounding a capable general model in a curated body of authoritative product knowledge that you control. The interesting part is that the body of knowledge required to make a model genuinely competent in a specific product is, for most enterprise vendors, already public. Administration guides, release notes, knowledge centre articles, support bulletins, hardening guides, command references — all of it sits on the vendor’s documentation site, free to read and free to index.

This post walks through a working proof-of-concept I built around exactly that idea:the BigFix Workspace+ Expert Agent, a Retrieval-Augmented Generation (RAG) specialist trained on public BigFix documentation. It is small enough to run on a laptop and good enough at its domain to demonstrate where this pattern earns its keep. More importantly, the architecture is product-agnostic — swap the knowledge base, change the system prompt, and the same pattern produces a specialist for VMware Horizon, ServiceNow ITSM, Citrix DaaS, Microsoft Intune, or any other vendor whose docs you can legally crawl.

Why General LLMs Are Not Good Enough For Product Depth

A general-purpose LLM pushed into vendor-specific territory tends to fail in the same five ways every time. A RAG-based specialist agent attacks all five at once — the knowledge base sits outside the model, can be refreshed when the vendor publishes a new release, retrieves verbatim passages instead of synthesising from memory, returns the source documents alongside every answer, and refuses to leave its declared domain.

The Premise — Public Vendor Data Is The Training Set

Enterprise software vendors publish an enormous amount of high-quality technical content for one reason: their customers cannot deploy or operate the product without it. HCL, Microsoft, VMware, Citrix, Red Hat, ServiceNow — all of them maintain extensive documentation sites and most also publish support knowledge bases, community forums, and architecture reference papers.

Three categories matter most for a support and architecture agent:

The first isproduct documentation — administration guides, deployment guides, console references, API references. This is your “how does the product work” layer. It is structured, versioned, and authoritative.

The second isknowledge base and support articles — the “how do you fix this when it breaks” layer. These articles are written by support engineers for symptoms that real customers have actually hit. They are gold for a troubleshooting agent.

The third isarchitecture and best-practice content — reference architectures, sizing guides, hardening guides, integration playbooks. This is your “how should you actually deploy it” layer.

Once you have those three corpora indexed, you have something that no general LLM has natively: a tight, current, citable body of knowledge with the depth that real practitioners need.

The Case Study — BigFix Workspace+ Expert Agent

HCL BigFix Workspace+ is a unified endpoint management platform — agent-based endpoint discovery, patch management for Windows / macOS / Linux / 100+ third-party apps, software distribution, CIS and STIG compliance scanning, and workspace analytics. It is a serious enterprise product and a representative target: the kind of platform where a general LLM is only superficially useful and where a specialist agent has obvious value for L1/L2 support, internal enablement, and architecture sanity-checks.

The POC ingests a curated set of public BigFix Workspace+ documentation across six domains:

1. Product overview and architecture
2. Patch management
3. Troubleshooting
4. Security and compliance
5. Software distribution
6. Workspace analytics

Each domain is a markdown document. In the POC the documents are hand-curated extracts; in a production deployment you would point the same pipeline at the vendor’s Knowledge Center, scrape it on a schedule, and feed the output straight through.

The Tech Stack along with production Swap:

Cost to run a demo session sits well under ten cents in API calls. The whole system fits in a single repository and runs on a laptop with a Python virtual environment.

Architecture

The pipeline is the standard RAG topology, split into an offline ingestion stage (runs when the knowledge base is built or refreshed) and a runtime query stage (runs for every user question). The vector store is the shared component that bridges the two.

The Ingestion Pipeline

Three steps: load, chunk, embed.

Loading is mechanical — read every markdown file in the knowledge base directory, attach source metadata (filename plus adomain tag so multiple products can share a database later), and return a list of documents.

Chunking is where most of the retrieval quality is actually won or lost. The POC uses LangChain’sRecursiveCharacterTextSplitter with separators ordered from most to least semantic — heading-level breaks first, then paragraph breaks, then sentences, then words:

splitter=RecursiveCharacterTextSplitter(chunk_size=1000,chunk_overlap=200,separators=["\n## ","\n### ","\n#### ","\n\n","\n",". "," "],)

The reason for this ordering matters. Technical documentation is dense with structured headings — “Creating a Patch Action”, “Patch Discovery”, “Maintenance Windows” — and a chunk that respects those boundaries reads as a coherent unit. A chunk that arbitrarily splits a numbered procedure in half does not. The 200-character overlap is there so that a sentence at the boundary still appears in both chunks; this materially improves recall when a user’s query phrasing matches text near a chunk edge.

Embeddings are produced withtext-embedding-3-small — fast, cheap, and accurate enough for technical English. For a knowledge base of around six hundred lines of markdown the entire ingestion runs in under five seconds and produces roughly eighty to a hundred chunks.

The Agent

Two things make the agent specialised: thesystem prompt and theretrieval contract.

The system prompt establishes the agent’s persona, the rules of engagement, and — critically — the refusal policy. The relevant parts:

You are the BigFix Workspace+ Expert Agent, a specialist AI assistant forHCL BigFix Workspace+ — a unified endpoint management platform.Rules:1. ONLY answer questions related to HCL BigFix Workspace+. If a question is outside your domain, politely redirect.2. Base your answers on the provided context. If the context doesn't contain enough information, say so honestly rather than guessing.3. When providing steps or commands, be specific and include the exact paths, commands, or navigation instructions.4. Always cite which area of BigFix your answer relates to.

The agent is told, explicitly, not to extrapolate. This is the lever that turns a chatty general-purpose model into something that behaves like a knowledgeable but appropriately humble support engineer. If the retrieved context does not address the question, the model is instructed to acknowledge the limitation rather than synthesise an answer from its parametric memory.

The retrieval contract is the second half of the specialisation. For every query, the agent retrieves the top-k chunks (k=4 in the POC) above a relevance threshold of 0.3, concatenates them into a single context block, and passes the question and context into the LLM. If retrieval returns nothing above threshold, the agent short-circuits and returns a polite “I do not have that in my knowledge base” response without ever calling the LLM. This is a small detail with a large effect — it prevents the model from confabulating answers when it has no evidence.

Confidence Scoring And Escalation

One of the design decisions I am most happy with is the explicit confidence score on every response. After generating an answer, the agent makes a second LLM call asking the model to rate how well the retrieved context actually addressed the question, on a scale of zero to one. If the score falls below a configurable threshold (default 0.7), the response carries an “escalation recommended” flag — surfaced in the UI as a coloured badge and, in production, used to route the question to a human L2 engineer or to a higher-tier orchestrator agent.

The result is a system that fails gracefully. It says “I don’t have that in my knowledge base” when retrieval comes back empty, and it raises a hand when retrieval succeeded but the response did not quite land. Very few naive RAG systems have either of those properties — most will confidently invent an answer either way.

Domain Guardrails

Ask the agent “what is the weather today” and it will not answer — not because it cannot, but because it has been instructed not to. The system prompt enforces a hard scope: BigFix Workspace+ questions only, everything else is politely redirected.

This sounds like a small thing. It is not. Domain isolation is the property that makes a specialist agenttrustworthy in an enterprise setting. A general assistant that wanders out of its lane is a liability — it will eventually answer a question that the organisation needs answered by a different specialist team, or worse, by a human with accountability. A specialist agent that refuses to leave its lane is the agentic equivalent of a senior engineer who knows what they know and is honest about what they do not.

This pattern also composes. Run a BigFix agent, a ServiceNow agent, an Intune agent, and a VMware Horizon agent in the same workspace, and give an orchestrator agent the job of routing incoming questions to whichever specialist is best placed to answer. The result is something that starts to look like a real virtual support organisation.

What This Approach Actually Buys You Over A General LLM

Five things, concretely.

The first iscurrency. The knowledge base can be refreshed on whatever cadence the vendor publishes new content — daily for an actively maintained product, weekly for everything else. The model itself never has to be retrained. Re-running ingestion against an updated set of source documents is the entire change-management process.

The second isaccuracy on product specifics. The answers come from passages that the vendor actually wrote, retrieved verbatim and handed to the LLM as context. The model still synthesises the response, but it is synthesising from authoritative source material rather than reaching into its compressed memory.

The third isauditability. Every response in the POC includes the list of source chunks that informed it, with relevance scores. A user — or an internal audit — can click into the sources and verify the claim against the underlying documentation.

The fourth isscope control. A general LLM will answer any question; a specialist agent answers only questions inside its declared domain. This is a feature, not a limitation. It is what makes the agent safe to deploy to a wide internal audience without needing to babysit its outputs.

The fifth isgraceful failure. The combination of a relevance threshold on retrieval, an explicit “I don’t have that in my knowledge base” response when nothing scores above the threshold, and a confidence score on every answer means the agent fails by saying “I don’t know” rather than by inventing an answer.

From POC To Production

The POC deliberately stops short of production. Six things would have to change to take this into an enterprise environment, and none of them are difficult — they are all standard enterprise-platform substitutions.

The first is theLLM provider. Swap OpenAI for Azure OpenAI to get enterprise compliance, data residency, and an enterprise SLA. The LangChain code already supports both — flipping between them is an environment-variable change.

The second is thevector store. ChromaDB is excellent for local development; for production you want Azure AI Search or Pinecone, both of which give you horizontal scale, snapshotting, and managed availability. The retrieval interface is identical.

The third isauthentication and access control. The Flask web UI is replaced with an Azure Bot Framework bot connected to MS Teams. The bot inherits Azure AD identity, which means every query is now tied to a named user, and access can be scoped per agent.

The fourth isobservability. Add LangSmith or a custom logging layer to capture query, retrieved chunks, confidence, response, and user identity for every interaction. This becomes the dataset that drives knowledge-base improvement — every time the agent says “I don’t know”, that is a gap to fill.

The fifth isautomated knowledge-base updates. Replace the manualpython ingest.py invocation with a scheduled pipeline that crawls the vendor’s documentation site, diffs against the last snapshot, and reindexes only what changed.

The sixth is themulti-agent layer. Once you have a second specialist agent (say, ServiceNow ITSM), you introduce an orchestrator agent — built on LangGraph — whose job is to read the incoming question, decide which specialist is best placed to answer, and route accordingly. This is where the architecture starts to earn its name as an “agentic library”.

Generalising The Pattern

The thing I want to draw out here is that almost nothing in the architecture above is specific to BigFix. The system prompt names the product, the knowledge base contains BigFix documentation, and the UI says “BigFix Workspace+ Agent” at the top. Everything else — the ingestion pipeline, the retrieval logic, the confidence scoring, the escalation flag, the domain guardrails — is the same for any vendor whose documentation you can index.

That is it. The same six steps produce a Citrix DaaS architecture agent, a ServiceNow ITSM workflow assistant, a VMware Horizon troubleshooting bot, a Microsoft Intune compliance helper, a Red Hat OpenShift operator’s companion. Pick the product, find the docs, run the pipeline.

Closing Thought

The interesting shift here is not technical. The technical pattern — retrieval-augmented generation — has been in production at large enterprises for two years. What is interesting is thatthe data you need to make a specialist agent meaningfully better than a general LLM at a specific vendor product is already free and public. The vendor put it there because their customers need it to operate the product. We have spent twenty years building search engines, knowledge bases, and support portals on top of that documentation. A specialist agent is the natural next layer — one that reads those same documents, understands the question being asked, and answers in the form the user actually wanted.

For me, the BigFix POC is one node in a larger experiment around what an agentic IT support and architecture function could look like. A specialist agent per product, an orchestrator on top, a confidence-aware escalation path into human L2 / L3 — all of it grounded in the documentation that already exists. None of the individual pieces are exotic. The interesting thing is what happens when you assemble them.

If you operate a workplace or infrastructure team and you have a product domain where the same questions get asked over and over by the same engineers, this is a weekend’s worth of work. The vendor has already written your training set.

Automation isn’t coming. It’s already here — and it’s accelerating faster than most organisations have planned for.

In the past two years, generative AI has moved from a novelty to a core part of how knowledge work gets done. Writing, coding, analysis, summarisation, customer interaction — tasks that once required specialists can now be initiated with a prompt. The World Economic Forum estimates that85 million jobs will be disrupted by 2025, while97 million new roles will emerge that require humans and machines to work together.

The uncomfortable truth is that the skills that made someone valuable in 2020 are not sufficient for 2030. Not because the work is disappearing entirely — but because the baseline has shifted. The tasks AI handles well are the ones most easily measured, most easily automated. What remains irreducibly human — and what organisations are now actively hiring for — are harder to fake, harder to code, and harder to scale.

The research is consistent across sources: theWorld Economic Forum Future of Jobs 2025 report, McKinsey’s workforce transition analyses, and a growing body of peer-reviewed evidence all point to the same cluster of capabilities. Not technical skills alone. Not soft skills alone. A specific combination of cognitive, interpersonal, leadership, and AI-fluency skills that together define effectiveness in an AI-augmented world.

This article walks through all 17 of them — drawn from a research-validated framework built on those primary sources — with enough detail to understand what each skill actually means and where to start building it.

The Framework: Four Categories, 17 Skills

The 17 skills fall into four categories:Cognitive Skills,Interpersonal & Human Skills,Leadership & Character Traits, andTechnical AI Skills. The split is intentional — effective performance in an AI-integrated workplace requires all four working together. Technical fluency without critical thinking produces prompt-execution without judgment. Interpersonal strength without resilience collapses under constant change. Leadership without integrity erodes trust at exactly the moment organisations need it most.

Part 1 — Cognitive Skills

Five cognitive skills define how you process information, communicate insight, and keep pace with change. These are the mental habits that determine whether you use AI as a tool or become dependent on it without understanding what it produces.

Critical Thinking

Critical thinking is the ability to analyse information objectively, identify assumptions, evaluate evidence, and reach reasoned conclusions — independent of what an AI, a manager, or a source claims. In an environment where AI can produce confident-sounding content that is factually wrong, this skill is no longer optional. The person who can interrogate an AI output, spot the flaw in a logical chain, and decide what actually holds up is the person who adds value above the model.

Storytelling

Data without narrative is noise. Storytelling is the capacity to take complex, technical, or ambiguous information and shape it into a form that moves people to understand and act. AI can draft a summary — it cannot yet understand which detail changes a room. The ability to read an audience, choose the right frame, and make information land is a persistent human advantage.

Learnability

Learnability is the drive and ability to continuously acquire new skills as conditions change. The half-life of technical knowledge is shortening — what was cutting-edge 18 months ago is now a commodity. The differentiator is not what you currently know but how quickly you can absorb, apply, and discard what you need. This is a learnable habit in itself: building structured routines around learning, reflection, and experimentation.

Financial Literacy

Financial literacy in this context means understanding the business logic behind decisions — how resources are allocated, how ROI is calculated, what the cost of inaction looks like. As AI tools proliferate, more people in non-finance roles are being asked to build business cases, evaluate vendor proposals, and justify investment. The person who can read a P&L and understand what it signals is better positioned to influence outcomes.

Metacognition

Metacognition is thinking about your own thinking. It’s the ability to notice when your reasoning is flawed, when a cognitive bias is operating, when you’re avoiding an uncomfortable conclusion. In fast-moving environments with incomplete information, this self-awareness separates people who course-correct early from those who double down on bad decisions. It’s also the foundation for using AI well — recognising where your own knowledge ends and where you need to verify.

Part 2 — Interpersonal & Human Skills

Five interpersonal skills define how you operate in relationship with others — colleagues, clients, teams, and the people impacted by your decisions. These are the skills most resistant to automation because they depend on presence, trust, and genuine human connection.

Human Behaviour

Understanding human behaviour — what motivates people, what creates resistance, how group dynamics operate — is the basis for effective collaboration, change management, and leadership. AI can analyse sentiment at scale; it cannot read the room in a difficult meeting or sense when a team is quietly disengaged. This skill is about developing the perceptual vocabulary to see what’s actually happening between people.

Empathy & Compassion

Empathy is not softness — it is a strategic capability. The ability to genuinely understand another person’s perspective, priorities, and constraints leads to better decisions, stronger relationships, and more effective communication. In a world where AI handles routine interactions, the interactions that remain human are the ones that matter most — and they require real presence.

Feedback Fluency

Feedback fluency is the ability to give, receive, and act on feedback without ego interference. Most organisations say they value feedback but have cultures where it rarely happens well. The people who can deliver honest assessment with care, receive difficult feedback without defensiveness, and build feedback loops into everyday work are significantly more effective — and make the teams around them more effective too.

Emotional Vulnerability

Emotional vulnerability is the willingness to admit uncertainty, acknowledge mistakes, and show up as a full human being in professional contexts. This is different from oversharing. It’s the courage to say “I don’t know,” “I got that wrong,” or “this is hard” — and in doing so, create the psychological safety that allows teams to do their best work. AI cannot model vulnerability. Leaders who can are rare and disproportionately effective.

Conflict Resolution

Conflict is not the problem — unresolved conflict is. The ability to navigate disagreement, surface the underlying interests beneath stated positions, and find workable outcomes without destroying relationships is essential in any organisation dealing with change. As AI accelerates the pace of change, the frequency and intensity of conflict around priorities, roles, and resources will increase. The people who can move through it constructively are invaluable.

Part 3 — Leadership & Character Traits

Five leadership traits define the character dimension of effectiveness. These are not competencies you can train in a weekend — they develop through sustained practice, reflection, and deliberate choices over time. They are also increasingly rare, and therefore increasingly valuable.

Patience

Patience is the capacity to maintain focus and composure in the face of delays, ambiguity, and setbacks — without becoming disengaged or reactive. In environments of rapid change, impatience is expensive: it leads to premature decisions, burned relationships, and abandoned initiatives just before they would have worked. Patience is not passivity; it’s the ability to sustain effort through the parts of the process that feel unproductive.

Resilience

Resilience is the ability to recover from setbacks, adapt to change, and keep going under sustained pressure — without losing effectiveness or identity. The pace of change in AI-adjacent industries means that failure, revision, and disruption are constant features of the work. Resilience is what separates people who iterate from those who stall. It can be built: through deliberate recovery practices, social support, and a relationship with failure that treats it as information rather than verdict.

Self-Reflection

Self-reflection is the regular practice of examining your own behaviour, decisions, and impacts — not to judge but to learn. Leaders who reflect are more consistent, more honest about their limitations, and more capable of genuine growth. In an environment where AI can surface patterns in your work that you would never notice, the people with strong self-reflection practices will use those insights productively rather than defensively.

Public Service Drive

Public service drive is the motivation to contribute beyond personal gain — to the organisation, the community, or the broader world. This is increasingly relevant as AI raises the stakes for decisions about automation, displacement, and the ethical use of technology. People who are driven by purpose beyond their own advancement make different choices, communicate differently, and attract different levels of trust from the people around them.

Integrity

Integrity is the alignment between your values and your actions — and the willingness to maintain that alignment when it’s costly. As AI systems enable new forms of data manipulation, synthetic content, and opaque decision-making, integrity becomes a competitive differentiator. Organisations that can demonstrate trustworthiness will command a premium. The people inside those organisations who model integrity shape the culture that makes that trustworthiness real.

Part 4 — Technical AI Skills

Two technical skills round out the framework. These are the most rapidly evolving — and the ones with the clearest immediate return on investment. Job postings referencing AI fluency rose7x between 2022 and 2024. The WEF projects 170 million new roles that require humans to collaborate effectively with AI systems. The floor for technical AI competence is rising.

Prompting & Vibe Coding

Prompting is the ability to communicate effectively with AI systems to get useful, accurate, and appropriately scoped outputs. This is not a trivial skill — it requires understanding how language models interpret instructions, how to provide context, how to structure multi-step tasks, and how to verify outputs critically. Vibe coding extends this to software development: describing what you want in natural language, iterating with an AI model, and producing working code without traditional programming expertise. Both capabilities are now teachable, testable, and directly applicable to most knowledge-work roles.

AI Workflow Integration

AI workflow integration is the ability to identify where AI can improve or replace steps in an existing process — and then actually implement that change. This requires both the technical literacy to understand what tools can do and the process thinking to redesign workflows around them. The people who are most effective here are not necessarily the most technically sophisticated; they are the ones who understand the work deeply enough to see where the leverage is.

Where to Start: Free Learning Paths

The final slide is a course directory — every course listed is free to audit on Coursera, mapped to the skill categories above. The path from “aware of these skills” to “actively developing them” starts with picking one. Not the most impressive one. The one that feels most relevant to where you’re working right now.

The Summary

The 17 skills above are not a checklist for becoming superhuman. They are a map of where the real work is — the capabilities that AI cannot replicate, that organisations are actively competing for, and that compound over time in ways that technical knowledge alone does not.

The division is useful:Cognitive skills shape how you think.Interpersonal skills shape how you connect.Leadership traits shape who you are under pressure.Technical AI skills shape how you extend your own capabilities. All four matter. The people who develop all four will have a meaningful advantage — not despite AI, but because of it.

The tools are available. The research is clear. The only question is where you start.

Sources: World Economic Forum Future of Jobs Report 2025; McKinsey Global Institute Workforce Transition Research; peer-reviewed literature on human skills in AI-augmented workplaces.

Credit: Idea for this presentation inspired by thispodcast.

Every architect has felt this at some point. You are mid-design on a complex Azure landing zone, you need a sanity check on your FSLogix profile container sizing, and the fastest path to an answer is to ping a colleague who knows AVD cold — except it is 10pm, or they are in another timezone, or that colleague simply does not exist in your organisation.

I built Aether to fix that. It is a local-first, multi-agent AI system that runs a team of 10 specialist IT architecture advisors on a single MacBook Pro M5. No internet after setup. No API costs. No data leaves the machine. Just fast, cited, domain-grounded answers — available at 10pm when the deadline hits.

This is the story of how I built it, what the stack looks like under the hood, and what I learned along the way.

The Problem I Was Actually Solving

I have spent years in IT architecture — cloud, digital workplace, network, end-user computing, the works. Over that time I have accumulated a large personal knowledge base: AWS Well-Architected reviews, Intune compliance policy templates, AVD host pool sizing guides, Citrix NetScaler configurations, TOGAF artefacts, cloud adoption frameworks. The knowledge exists. The problem is retrieval — getting the right answer from the right domain quickly, without context-switching across six different documentation tabs.

Commercial AI tools are good at general answers. They are not great at answering “give me the exact OMA-URI path for configuring Windows Hello for Business through Intune on a hybrid-joined device for a tenant with MFA enforced at the Conditional Access layer.” That requires domain depth, and it requires knowing which documents to pull from.

I also wanted to explore a practical AI use case — not a demo, not a proof of concept, but something I would actually use daily. Aether became that experiment.

What Aether Is

AETHER v2.6 // SYSTEM DEFINITION

Type:      Local-First Multi-Agent AI System
Agents:    10 (3-tier hierarchy)
Model:     Gemma 4 26B A4B (Q4_K_M) — single instance
Runtime:   LM Studio → LangGraph → FastAPI → Gradio
Memory:    ~43 GB of 64 GB unified (M5 Pro)
Egress:    ZERO
API cost:  ZERO

Aether is alocal-first AI system that acts as a team of specialist architecture advisors — all running on your MacBook Pro M5. The headline technical trick: every one of those 10 advisors is thesame Gemma 4 26B model, loaded once. What makes each advisor different is purely the system prompt it receives and the knowledge base namespace it retrieves from.

One model. Ten personas. Zero cloud.

---

The Three-Tier Agent Hierarchy

The agents are organised the way a real consulting firm would structure a team — from narrow specialist up to cross-domain strategist.

Tier 1 — Enterprise Architect (1 agent). Cross-domain strategy, TOGAF, Zachman, governance frameworks (GDPR, ISO 27001, HIPAA), technology investment decisions. This agent can readall knowledge base namespaces — it is the only one with that reach. The final escalation destination.

Tier 2 — Domain Architects (3 agents). Cloud Domain (multi-cloud strategy, FinOps, landing zones), Network Domain (SD-WAN, ZTNA, BGP, micro-segmentation), Digital Workplace Domain (Microsoft 365, VDI strategy, device management).

Tier 3 — Technology Architects (6 agents). AWS, Azure, GCP, Intune, AVD, Citrix. Each one is scoped tightly to its domain — deep, narrow, and fast.

---

The Stack, Component by Component

Component	What It Does
Gemma 4 26B A4B (Q4_K_M)	The single model serving all 10 agents. MoE architecture — activates ~4–6B parameters per token. ~13 GB VRAM.
LM Studio	Local model server. OpenAI-compatible API on port 1234. Model stays resident in unified memory.
LangGraph	Orchestration graph — defines the multi-step query pipeline as a typed state machine.
LanceDB	Local vector database. One namespace (table) per agent. Fully file-based, no server process needed.
BAAI/bge-small-en-v1.5	Embedding model. 384-dimensional vectors. Runs on the Apple Neural Engine — essentially free compute.
Redis	Session memory (24h TTL, rolling 3-turn window) and routing cache (1h TTL).
FastAPI + Uvicorn	REST API gateway on port 8000. Full query/response model, session management, agent listing.
Gradio	Web chat UI on port 7860. Session management, source citations, escalation chain display.
Prometheus	Metrics: query counts, latency per agent, escalation rate. Useful for understanding usage patterns.
SQLite (audit.db)	Immutable audit trail. Every query logged before response is returned. Cannot be skipped.

The full system sits comfortably in about 43 GB of the M5 Pro’s 64 GB unified memory — leaving 21 GB of headroom for the rest of the machine.

---

The “One Model, Ten Specialists” Trick

This is the part I get asked about most, because it sounds like it should not work.

Every agent in Aether is defined by a YAML manifest. That manifest specifies asystem_prompt, anamespace (which LanceDB table to retrieve from), atemperature, andmax_tokens. There is no model switching. There is no weight loading. The Gemma 4 26B A4B is loaded once by LM Studio and stays resident.

# agent_manifests/aws_technology_architect.yamlagent_id:aws_technology_architectdisplay_name:"AWS Technology Architect"tier:3parent_agent:cloud_domain_architectnamespace:aws_techtemperature:0.1max_tokens:2048system_prompt:| You are an AWS Technology Architect with deep expertise in the AWS Well-Architected Framework, EC2/EKS/Lambda sizing, IAM policy design, CloudFormation and CDK, GuardDuty, and cost optimisation. You draw only from AWS-specific documentation and architecture patterns. When answering, cite the source documents retrieved. End your response with a new line: 'Confidence: X.XX' (0.00–1.00) reflecting how well your knowledge base supports this answer.

Why does this work? Because Gemma 4 26B A4B is a Mixture of Experts model. It routes each token through specialist sub-networks internally — activating only about 4–6 billion parameters per inference pass, despite having 26 billion total. The practical result: it runs at near 5B speed while retaining the reasoning breadth of a much larger dense model. A single loaded instance can faithfully adopt both a narrow Citrix specialist persona and a broad enterprise strategy persona, because the MoE routing shifts for each.

The other half of the trick is the knowledge base. Each Tier 3 agent retrieves only from its own LanceDB namespace. The AWS agent never sees a Citrix document. The Intune agent never sees a GCP architecture guide. Domain knowledge is isolated by design — which means the model cannot hallucinate across domain boundaries, because the retrieval context does not cross them.

---

Confidence-Driven Escalation

The escalation mechanism is the design decision I am most proud of.

Every agent is instructed to append a confidence score to its response — a float between 0.00 and 1.00 representing how well its retrieved knowledge supports the answer. The orchestrator reads that score via regex. If it falls below 0.7and the agent has a parent tier definedand that parent has not already been tried, the system automatically escalates.

// ESCALATION CHAIN EXAMPLE

Query: "Give me a multi-cloud strategy covering Azure, AWS, GCP, AVD, and Citrix with network segmentation"

→ Routed to:aws_technology_architect
→ Confidence:0.41 (below 0.7 threshold)
→ Escalate to:cloud_domain_architect
→ Confidence:0.63 (still below threshold)
→ Escalate to:enterprise_architect
→ Confidence:0.88 ✓

Response appended with escalation chain for full transparency.

The response shown to the user includes the full escalation path — which level of expertise produced the final answer. This matters in enterprise contexts. It is the difference between “the AI said so” and “the enterprise-level advisor produced this after the technical specialist’s knowledge was insufficient.”

The elegance here is thatthe model participates in its own routing decision. The orchestration does not need a separate classifier to judge answer quality — the model tells you its own confidence, and the system acts on it.

---

The 7-Step Query Pipeline

Every query passes through a typed LangGraph state graph. The state object —AetherState — carries everything between nodes: query, session ID, agent manifest, RAG results, conversation history, messages, response, confidence score, escalation flags, and escalation chain list.

01 — ROUTE. A keyword scanner maps the query to the best-fit agent, checking Tier 3 rules first (most specific), then Tier 2, then Tier 1 as catch-all. Redis caches route results for one hour, so repeated queries on the same topic skip the scan entirely.

02 — RETRIEVE. Semantic search against that agent’s LanceDB namespace — top-5 chunks returned. Documents were ingested at 500-word chunks with 50-word overlap, embedded into 384-dimensional vectors byBAAI/bge-small-en-v1.5 running on the Apple Neural Engine.

03 — HISTORY. The last three message pairs (six messages) are loaded from Redis for the session. This gives conversational continuity without letting the context window balloon.

04 — BUILD. The message payload is assembled:system_prompt + RAG documents + history + current query. The confidence instruction is appended here.

05 — LLM. The assembled payload hits LM Studio on port 1234. The orchestrator extracts the confidence score from the response text via regex before passing the response forward.

06 — ESCALATE (conditional). If confidence is below 0.7, a parent agent exists, and it has not already been tried this turn — swap in the parent’s manifest and loop back to step 02 with fresh retrieval against the parent’s namespace.

07 — FINALISE. The turn is saved to Redis. The SQLite audit record is written. The escalation chain annotation (if any) is appended to the response. Result returned to the user.

---

Knowledge Isolation — The Anti-Hallucination Architecture

One of the most practical decisions in Aether’s design is namespace isolation. Each Tier 3 agent retrieves only from its own LanceDB table. The AWS agent’s retrieval context will never include a Citrix StoreFront configuration guide — because those documents simply do not exist in its namespace.

This matters more than it might seem. A common failure mode in RAG systems iscross-domain contamination — where retrieval pulls in tangentially related content from a different domain, and the model confabulates a plausible-sounding but wrong answer by blending the two. Namespace isolation eliminates this at the architectural level.

The Enterprise Architect at Tier 1 is the deliberate exception — it can query all namespaces, because cross-domain synthesis is exactly what it is built for.

---

The Audit Trail — Because Enterprise

Every single query is written toaudit.db (SQLite) before the response is returned. The record includes: timestamp, session ID, query text, agent used, confidence score, escalation chain, and sources cited. The audit write is wrapped in error handling so that a database failure never blocks the main query flow — but the log is never optional.

# Simplified from the finalise nodeaudit_record={"timestamp":datetime.utcnow().isoformat(),"session_id":state["session_id"],"query":state["query"],"agent_id":state["agent_id"],"confidence":state["confidence"],"escalation_chain":json.dumps(state["escalation_chain"]),"response_length":len(state["final_response"])}db.execute("INSERT INTO audit_log VALUES (:timestamp, :session_id, ...)",audit_record)

For a system giving architecture recommendations — decisions that feed into multi-million dollar cloud commitments — having an immutable audit trail of what was asked, what agent answered, at what confidence level, via what escalation path, is not a nice-to-have. It is the thing that makes it organisationally defensible.

---

What It Can Actually Do

$aether query"Design an Azure landing zone for PCI-DSS compliance"
$aether query"Right-size EC2 instances for a memory-intensive Java application"
$aether query"Configure FSLogix profile containers for 500 AVD users"
$aether query"Zero-trust network architecture for 5,000 remote employees"
$aether query"Cloud adoption roadmap for a financial services firm"
$aether query"Intune compliance policy for iOS BYOD — hybrid-joined, MFA enforced"

The depth per domain is real. The Intune agent knows OMA-URI paths, Graph API commands, Autopilot profiles, and licensing requirements — because those are the documents I ingested into its namespace. The AVD agent knows host pool design, FSLogix sizing, MSIX app attach, and session host scaling plans. The knowledge base is only as good as what you put into it — but that is also the point. This ismy architecture knowledge, curated, searchable, and queryable at any hour.

---

What I Learned Building This

Prompt engineering IS the architecture. In a system like this, the YAML manifestis the agent. The difference between a brilliant AWS specialist and a generic AI assistant is entirely in what the system prompt says and what documents back it up. Getting those prompts precise, domain-bounded, and calibrated for the right temperature took longer than any of the code.

MoE models are underrated for local multi-agent work. The choice of Gemma 4 26B A4B over a dense model was the right call. You get reasoning breadth comparable to a much larger model at the inference cost of a small one, on hardware that most architects already carry.

Confidence as a first-class citizen. Asking the model to self-assess and surface that score is one of the highest-value things I added. It makes the system honest — and it drives the escalation logic that makes the team metaphor actually work.

Namespace isolation is a practical hallucination brake. Not a theoretical one. In the first version of Aether, all documents lived in a single namespace. Cross-domain contamination was visible and annoying. Splitting into per-agent namespaces fixed it immediately.

Audit trails are not overhead — they are the point. Every enterprise AI deployment should have one. Building it into the core pipeline from day one changes how you think about what the system is producing.

---

What Is Next

Aether v2.6 is a working, daily-use system. The next version I am working toward adds a web-based ingestion UI (so loading new documents does not require touching the ingest script), structured output for architecture decision records (ADRs) in a consistent format, and inter-agent communication — where a Tier 3 agent can proactively pull context from a peer rather than waiting for the escalation chain to activate.

I built Aether because I wanted a team. It turns out a team was always available — it just needed the right prompts and a local model to bring it to life.

---

Have questions about the stack or want to see specific parts of the implementation? Drop a comment or reach out onLinkedIn.

]]>Postsartificial-intelligencearchitectureautomationengineeringllmKnowledge Basehttps://curiousbit.netlify.app/i-built-my-own-video-downloader-no-ads-no-watermarks-three-platforms/https://curiousbit.netlify.app/i-built-my-own-video-downloader-no-ads-no-watermarks-three-platforms/Mon, 11 May 2026 00:00:00 +0000Ajay Walia<p>Every video download site I tried felt like navigating a minefield — five ad clicks to reach a button that triggers another redirect. So I stopped using them and built a clean local tool that handles TikTok, Instagram, and X in one paste.</p><![CDATA[

Every video download site I tried felt like navigating a minefield — five ad clicks to reach a button that triggers another redirect. So I stopped using them and built a clean local tool that handles TikTok, Instagram, and X in one paste.

No watermarks. No ads. No accounts. Just a URL.

~6 min read · Node.js + React · conceptual deep-dive

The Problem With Every Downloader Site

You find a TikTok video you want to keep. Maybe it's a tutorial, a recipe, a clip you want to share somewhere offline. You Google "download TikTok without watermark" and click the first result. What follows is a ritual:

Pop-up #1 appears. You close it. A second tab opens. You close that. You find the actual download button — but it's fake and triggers another ad. The real button is somewhere underneath a consent banner. You finally click download. It starts… and gives you a watermarked file anyway.

This happens on nearly every popular downloader site. They're monetised almost entirely through advertising, and the UX is designed to maximise your exposure to that advertising — not to help you download a video. The actual download logic underneath all that noise is usually a single API call.

So I asked myself: how hard would it actually be to build a clean version of this for personal use?

What I Actually Wanted

Three things, nothing more:

1

No watermark, always

The tool should try its hardest to get a watermark-free file. If it can't, it tells you — it doesn't quietly give you the watermarked version pretending it's clean.

2

Support the platforms I actually use

TikTok, Instagram, and X (Twitter). Paste any URL from any of these three and it should just work.

3

Zero friction interface

One input, one button. No accounts, no CAPTCHAs, no ads. It runs locally so there's nothing to sign up for.

How It Works — The Conceptual Picture

The tool is split into two halves: a lightweight React frontend and a Node.js backend. The frontend is just the URL input box. All the interesting logic lives in the backend.

Here are the five things that happen the moment you paste a URL and hit Download:

Step 1 — Figure Out Which Platform You're On

The very first thing the backend does is inspect the URL and decide: is this TikTok, Instagram, or X? Each platform has its own URL patterns — including short links, mobile URLs, and regional variants — and the tool checks all of them.

tiktok.com · vm.tiktok.com · vt.tiktok.com

instagram.com · www.instagram.com

x.com · twitter.com · mobile.twitter.com

If the URL doesn't match any of these, the backend rejects it immediately with a clear error before wasting any time trying to fetch something it can't handle. No silent failures.

The backend runs through a short chain: detect → resolve → try providers → cache → serve

Step 2 — Resolve Short Links

TikTok in particular loves to generate short share links likevm.tiktok.com/AbcXyz. These redirect to the full video URL, but the download providers need the real URL to work with. So the backend follows up to five redirects to resolve the final destination before doing anything else.

Step 3 — The Provider Chain

This is the core of the tool. The backend doesn't rely on a single source for the download link — it tries multiple providers in order, and only moves to the next one if the previous failed.

Validated & resolved URL

↓

Provider 1: TikWM API — fast, HD, usually no-watermark

↓ if failed

Provider 2: yt-dlp — catches what TikWM misses, supports all 3 platforms

↓ if both fail on no-watermark

Last resort: watermarked fallback (with a clear warning shown)

↓

✓ Download link returned to frontend

The first provider, TikWM, is a public API that's fast and usually returns an HD, watermark-free file. But it occasionally struggles with newer videos or private content. That's whenyt-dlp steps in — a powerful open-source tool that knows how to extract media from hundreds of platforms, and is updated constantly as platforms change their serving behaviour.

Why yt-dlp as a fallback and not primary? TikWM is faster and returns a clean pre-parsed result. yt-dlp is more capable but adds latency since it runs as a local process and parses raw platform data. Using TikWM first keeps the happy path quick.

Step 4 — The Proxy Download

Here's a detail that actually matters for reliability: the tool doesn't give your browser a direct link to TikTok's CDN or Instagram's servers. Instead, it registers a short-livedsecure token that points back to the Node backend. When you click download, your browser hits the backend's own/api/file endpoint, which streams the video directly to you.

Why does this matter? Direct CDN links from social platforms often include authentication tokens or short expiry times. They also sometimes block downloads when accessed directly from a browser outside the platform. Running the stream through the backend sidesteps both issues — and means the download starts with a clean filename instead of a jumble of CDN parameters.

Step 5 — Caching

Once a video URL has been resolved and a download link extracted, the result is cached in memory for 30 minutes. If you (or someone else on the same local instance) pastes the same video URL again within that window, the backend returns the cached result instantly — no API calls, no yt-dlp process, just the stored answer.

The Result

What this adds up to in practice:

🎯

No-watermark first, always

The tool tries every avenue for a clean file before falling back. The fallback is clearly labelled.

⚡

Fast on repeat URLs

Same video twice within 30 minutes? Instant response from the in-memory cache.

🎵

Audio extraction too

The tool also surfaces the audio-only track when available — useful for saving music from TikTok.

🔒

No external accounts

Runs entirely locally. Nothing to log into, nothing phoning home, no API keys required.

One URL input, three download buttons, zero pop-ups. That's the whole interface.

What's Next

The tool is currently running locally but I'm planning to deploy it oncuriousbit.netlify.app so anyone who wants a clean download experience can use it without having to run Node themselves. The architecture is already production-ready — it's just a matter of pointing it at a hosting environment and wiring up the environment variables.

A few things I'd like to add before making it fully public: rate limiting per IP (to avoid abuse), a simple download history in the UI, and potentially Instagram Stories support which currently needs a different extraction path.

The code is structured as a monorepo — backend and frontend live together, share URL validation logic, and build to a single deployable package. If you want to run it yourself locally, it's a singlenpm install && npm run dev away.

---

Over to you: Have you ever got fed up enough with a broken web experience that you built your own alternative? I'd love to hear what you made — or whether you'd actually use a clean, ad-free downloader like this if it were publicly hosted. Drop your thoughts below or find me on LinkedIn.

]]>Postsautomationbuild-lognodejsKnowledge Basehttps://curiousbit.netlify.app/attention-is-all-you-need-the-paper-that-rewired-ai/https://curiousbit.netlify.app/attention-is-all-you-need-the-paper-that-rewired-ai/Sun, 10 May 2026 00:00:00 +0000Ajay Walia<style> @import url('https://fonts.googleapis.com/css2?family=Bangers&family=Space+Grotesk:wght@400;500;600;700&family=Inter:wght@400;500&family=JetBrains+Mono:wght@400;500&display=swap'); .attn-article { --bg: #080b14; --surface: #111827; --surface2: #1a2235; --border: #1f2d45; --text: #e2e8f0; --muted: #8b9ab3; --purple: #8b5cf6; --purple-glow: rgba(139,92,246,0.2); --cyan: #22d3ee; --cyan-glow: rgba(34,211,238,0.15); --gold: #f59e0b; --gold-glow: rgba(245,158,11,0.2); --red: #ef4444; --green: #10b981; --manga-bg: #0e1520; --manga-border: #f59e0b; } /* ── Particles background ── */ /* ── Site header ── */ /* ── Main wrapper ── */ /* ── Hero ── */ .hero { text-align: center; padding: 5rem 0 3rem; } .hero-tag { display: inline-block; background: var(--purple-glow); border: 1px solid var(--purple); color: var(--purple); font-family: 'Space Grotesk', sans-serif; font-size: 0.75rem; font-weight: 700; letter-spacing: 3px; text-transform: uppercase; padding: 0.35rem 1rem; border-radius: 100px; margin-bottom: 1.5rem; } .hero h1 { font-family: 'Bangers', cursive; font-size: clamp(2.8rem, 8vw, 5.5rem); line-height: 1.05; letter-spacing: 3px; background: linear-gradient(135deg, #fff 0%, var(--cyan) 40%, var(--purple) 100%); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text; margin-bottom: 1.2rem; } .hero-sub { font-size: clamp(1.15rem, 1rem + 0.5vw, 1.45rem); color: var(--muted); max-width: 980px; margin: 0 auto 2rem; line-height: 1.7; } .hero-meta { display: flex; align-items: center; justify-content: center; gap: 1.5rem; flex-wrap: wrap; font-size: 0.82rem; color: var(--muted); font-family: 'Space Grotesk', sans-serif; } .hero-meta span { display: flex; align-items: center; gap: 0.4rem; } /* ── Manga Panel ── */ .manga-panel { margin: 2.5rem 0; background: var(--manga-bg); border: 2px solid var(--manga-border); border-radius: 4px; padding: 1.5rem; position: relative; box-shadow: 0 0 30px rgba(245,158,11,0.1), inset 0 0 60px rgba(0,0,0,0.4); } .manga-panel::before { content: '🎨 GROK IMAGINE'; position: absolute; top: -1px; left: -1px; background: var(--manga-border); color: #000; font-family: 'Bangers', cursive; font-size: 0.8rem; letter-spacing: 2px; padding: 0.2rem 0.8rem; border-radius: 2px 0 4px 0; } .manga-panel .panel-title { font-family: 'Bangers', cursive; font-size: clamp(1.35rem, 1rem + 1vw, 2rem); letter-spacing: 2px; color: var(--gold); margin: 1rem 0 0.75rem; } .manga-panel img { width: 100%; border-radius: 6px; display: block; margin-top: 0.5rem; box-shadow: 0 4px 24px rgba(0,0,0,0.5); } section, article { font-size: clamp(1.15rem, 1.05rem + 0.3vw, 1.4rem); line-height: 1.85; } /* ── Chapter headings ── */ .chapter-label { font-family: 'Bangers', cursive; font-size: 0.85rem; letter-spacing: 4px; color: var(--cyan); margin-bottom: 0.5rem; display: block; } h2 { font-family: 'Space Grotesk', sans-serif; font-size: clamp(1.6rem, 4vw, 2.2rem); font-weight: 700; margin-bottom: 1.2rem; line-height: 1.2; color: #fff; } h2 .highlight { color: var(--cyan); } h3 { font-family: 'Space Grotesk', sans-serif; font-size: 1.2rem; font-weight: 600; color: var(--purple); margin: 2rem 0 0.75rem; } /* ── Body text ── */ p { margin-bottom: 1.2rem; color: var(--text); } strong { color: #fff; } em { color: var(--cyan); font-style: italic; } /* ── Section divider ── */ .section { margin-top: 4.5rem; padding-top: 1rem; border-top: 1px solid var(--border); } /* ── Callout boxes ── */ .callout { margin: 2rem 0; padding: 1.25rem 1.5rem; border-radius: 6px; border-left: 4px solid; } .callout.purple { background: var(--purple-glow); border-color: var(--purple); } .callout.cyan { background: var(--cyan-glow); border-color: var(--cyan); } .callout.gold { background: var(--gold-glow); border-color: var(--gold); } .callout-title { font-family: 'Space Grotesk', sans-serif; font-weight: 700; font-size: 0.85rem; letter-spacing: 1px; text-transform: uppercase; margin-bottom: 0.5rem; } .callout.purple .callout-title { color: var(--purple); } .callout.cyan .callout-title { color: var(--cyan); } .callout.gold .callout-title { color: var(--gold); } .callout p { margin: 0; font-size: 0.95rem; } /* ── Inline diagram: RNN chain ── */ .diagram-box { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 1.5rem; margin: 2rem 0; overflow-x: auto; } .diagram-title { font-family: 'Space Grotesk', sans-serif; font-size: 0.78rem; font-weight: 600; letter-spacing: 2px; text-transform: uppercase; color: var(--muted); margin-bottom: 1rem; text-align: center; } /* RNN Chain */ .rnn-chain { display: flex; align-items: center; gap: 0; min-width: 600px; justify-content: center; } .rnn-node { width: 52px; height: 52px; border-radius: 50%; border: 2px solid #ef4444; background: rgba(239,68,68,0.1); display: flex; align-items: center; justify-content: center; font-family: 'Space Grotesk', sans-serif; font-size: 0.7rem; font-weight: 600; color: #ef4444; flex-shrink: 0; position: relative; } .rnn-arrow { width: 32px; height: 2px; background: linear-gradient(90deg, #ef4444, rgba(239,68,68,0.3)); position: relative; flex-shrink: 0; } .rnn-arrow::after { content: ''; position: absolute; right: 0; top: -4px; border-left: 8px solid rgba(239,68,68,0.5); border-top: 5px solid transparent; border-bottom: 5px solid transparent; } .rnn-fade { opacity: 0.35; } .rnn-labels { display: flex; justify-content: space-between; font-size: 0.7rem; color: var(--muted); margin-top: 0.6rem; font-family: 'JetBrains Mono', monospace; min-width: 600px; } /* Attention Matrix */ .attn-matrix { display: grid; grid-template-columns: auto repeat(6, 1fr); gap: 3px; font-size: 0.7rem; font-family: 'JetBrains Mono', monospace; } .attn-label { display: flex; align-items: center; justify-content: flex-end; padding-right: 8px; color: var(--muted); font-size: 0.68rem; } .attn-col-labels { display: grid; grid-template-columns: auto repeat(6, 1fr); gap: 3px; margin-bottom: 3px; } .attn-col-label { text-align: center; color: var(--muted); font-size: 0.65rem; font-family: 'JetBrains Mono', monospace; } .attn-cell { height: 36px; border-radius: 3px; display: flex; align-items: center; justify-content: center; font-size: 0.65rem; font-weight: 600; transition: transform 0.2s; cursor: default; } .attn-cell:hover { transform: scale(1.08); } /* Timeline */ .timeline { position: relative; padding-left: 2rem; margin: 2rem 0; } .timeline::before { content: ''; position: absolute; left: 7px; top: 0; bottom: 0; width: 2px; background: linear-gradient(180deg, var(--purple), var(--cyan), var(--gold)); } .timeline-item { position: relative; margin-bottom: 2rem; padding-left: 1.5rem; } .timeline-dot { position: absolute; left: -1.85rem; top: 0.35rem; width: 16px; height: 16px; border-radius: 50%; border: 2px solid; background: var(--bg); } .timeline-dot.purple { border-color: var(--purple); box-shadow: 0 0 10px var(--purple); } .timeline-dot.cyan { border-color: var(--cyan); box-shadow: 0 0 10px var(--cyan); } .timeline-dot.gold { border-color: var(--gold); box-shadow: 0 0 10px var(--gold); } .timeline-dot.green { border-color: var(--green); box-shadow: 0 0 10px var(--green); } .timeline-year { font-family: 'Bangers', cursive; font-size: 1.2rem; letter-spacing: 2px; color: var(--gold); margin-bottom: 0.25rem; } .timeline-milestone { font-family: 'Space Grotesk', sans-serif; font-weight: 700; font-size: 1rem; color: #fff; margin-bottom: 0.25rem; } .timeline-desc { font-size: 0.88rem; color: var(--muted); line-height: 1.5; } .timeline-badge { display: inline-block; font-size: 0.65rem; font-weight: 700; font-family: 'Space Grotesk', sans-serif; padding: 0.15rem 0.5rem; border-radius: 100px; margin-left: 0.5rem; vertical-align: middle; } .badge-origin { background: rgba(139,92,246,0.25); color: var(--purple); border: 1px solid var(--purple); } .badge-encoder { background: rgba(34,211,238,0.2); color: var(--cyan); border: 1px solid var(--cyan); } .badge-decoder { background: rgba(245,158,11,0.2); color: var(--gold); border: 1px solid var(--gold); } .badge-scale { background: rgba(16,185,129,0.2); color: var(--green); border: 1px solid var(--green); } /* Citation / quote block */ blockquote { margin: 2rem 0; padding: 1.5rem 1.5rem 1.5rem 2rem; background: var(--surface2); border-left: 4px solid var(--purple); border-radius: 0 6px 6px 0; font-style: italic; color: var(--muted); position: relative; } blockquote::before { content: '"'; font-family: 'Bangers', cursive; font-size: 5rem; color: var(--purple); opacity: 0.3; position: absolute; top: -1rem; left: 0.5rem; line-height: 1; } blockquote cite { display: block; margin-top: 0.75rem; font-size: 0.82rem; font-style: normal; color: var(--purple); font-family: 'Space Grotesk', sans-serif; font-weight: 600; } /* Code-like inline */ code { font-family: 'JetBrains Mono', monospace; font-size: 0.85em; background: rgba(139,92,246,0.15); color: var(--purple); padding: 0.15em 0.4em; border-radius: 3px; } /* Problem list */ .problem-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 1rem; margin: 1.5rem 0; } .problem-card { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 1.25rem; transition: border-color 0.2s; } .problem-card:hover { border-color: var(--red); } .problem-card .icon { font-size: 1.8rem; margin-bottom: 0.6rem; } .problem-card h4 { font-family: 'Space Grotesk', sans-serif; font-size: 0.95rem; font-weight: 700; color: var(--red); margin-bottom: 0.4rem; } .problem-card p { font-size: 0.88rem; color: var(--muted); margin: 0; } /* Breakthrough cards */ .breakthrough-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 1rem; margin: 1.5rem 0; } .breakthrough-card { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 1.25rem; transition: border-color 0.2s, box-shadow 0.2s; } .breakthrough-card:hover { border-color: var(--green); box-shadow: 0 0 20px rgba(16,185,129,0.1); } .breakthrough-card .icon { font-size: 1.8rem; margin-bottom: 0.6rem; } .breakthrough-card h4 { font-family: 'Space Grotesk', sans-serif; font-size: 0.95rem; font-weight: 700; color: var(--green); margin-bottom: 0.4rem; } .breakthrough-card p { font-size: 0.88rem; color: var(--muted); margin: 0; } /* Attention heads visual */ .heads-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 0.75rem; margin: 1.5rem 0; } .head-card { padding: 1rem; border-radius: 6px; text-align: center; border: 1px solid; } .head-card .head-num { font-family: 'Bangers', cursive; font-size: 1.6rem; letter-spacing: 1px; } .head-card .head-desc { font-size: 0.72rem; font-family: 'Space Grotesk', sans-serif; margin-top: 0.25rem; } .head-1 { border-color: #8b5cf6; background: rgba(139,92,246,0.1); } .head-1 .head-num { color: #8b5cf6; } .head-1 .head-desc { color: #8b5cf6; } .head-2 { border-color: #22d3ee; background: rgba(34,211,238,0.1); } .head-2 .head-num { color: #22d3ee; } .head-2 .head-desc { color: #22d3ee; } .head-3 { border-color: #f59e0b; background: rgba(245,158,11,0.1); } .head-3 .head-num { color: #f59e0b; } .head-3 .head-desc { color: #f59e0b; } .head-4 { border-color: #10b981; background: rgba(16,185,129,0.1); } .head-4 .head-num { color: #10b981; } .head-4 .head-desc { color: #10b981; } /* Big stat */ .big-stat { text-align: center; padding: 2.5rem 1rem; background: var(--surface); border-radius: 12px; margin: 2rem 0; } .big-stat .number { font-family: 'Bangers', cursive; font-size: 4rem; letter-spacing: 3px; background: linear-gradient(135deg, var(--gold), var(--purple)); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text; } .big-stat .label { font-family: 'Space Grotesk', sans-serif; font-size: 0.9rem; color: var(--muted); margin-top: 0.5rem; } /* Footer */ /* Responsive */ @media (max-width: 600px) { main { width: 100%; } .site-nav-inner { gap: 1rem; } .site-brand span:last-child { display: none; } .site-nav .nav-links { gap: 0.9rem; overflow-x: auto; } .heads-grid { grid-template-columns: repeat(2, 1fr); } .hero h1 { font-size: 2.5rem; } } /* Scroll fade-in */ .fade-in { opacity: 0; transform: translateY(24px); transition: opacity 0.6s ease, transform 0.6s ease; } .fade-in.visible { opacity: 1; transform: translateY(0); } /* Inlined into Hugo: ensure content is visible even without the IntersectionObserver script */ .attn-article .fade-in { opacity: 1; transform: none; } </style> <div class="attn-article"> <!-- ── HERO ── --> <section class="hero fade-in"> <div class="hero-tag">Deep Dive · Artificial Intelligence · 2017</div> <p class="hero-sub"> The seven-word title that ended one era of AI and launched another. A beginner-friendly, technically honest tour through the paper that birthed every LLM you've ever heard of. </p><![CDATA[

Deep Dive · Artificial Intelligence · 2017

The seven-word title that ended one era of AI and launched another. A beginner-friendly, technically honest tour through the paper that birthed every LLM you've ever heard of.

✍️ Ajay Walia📅 May 2026⏱ ~15 min read🧠 Beginner → Intermediate

Panel 1 — THE SCROLL APPEARS

CHAPTER 01

TheDark Ages of Language AI

To understand why "Attention Is All You Need" was a thunderclap, you first need to appreciate how painful life was before it. Cast your mind back to 2016. AI researchers around the world were working incredibly hard on language problems — translation, summarisation, question answering — but they were doing so with a fundamental handicap baked into their tools.

The dominant models at the time wereRecurrent Neural Networks (RNNs) and their smarter cousin, theLong Short-Term Memory network (LSTM). Both were designed to handle sequences: text goes in word by word, the model builds up a hidden memory state as it reads, and produces an output at the end.

The intuition seems sensible. After all,we read left to right (in English). Why shouldn't a machine? The problem, as we'll see, was catastrophic at scale.

How an RNN Actually Works

Imagine you're a gold fish with a tiny little notepad. Every time you read a new word, you scribble something on your notepad, thenerase half of it to make room for the next word. By the time you reach the end of a 500-word paragraph, your notepad is a smeared mess of partial impressions. That's an RNN.

More precisely: an RNN processes tokensone at a time, left to right. At each step, it combines the current word's embedding with ahidden state vector (its "memory" from all previous words) and produces a new hidden state. That hidden state is passed forward to the next step.

RNN: Sequential Processing Chain

The

cat

sat

on

the

???

Step 1Step 2Step 3Step 4Step 5 (fading)Long range: ☠️

Information degrades as it passes along the chain. Early words become "forgotten."

LSTMs: A Better Notepad, Same Problem

LSTMs (invented by Hochreiter & Schmidhuber in 1997) were the RNN's upgrade. Instead of one hidden state, they have three "gates" — input, forget, and output — plus a separate "cell state" that acts as a longer-term memory. They were genuinely better at remembering things across longer sequences.

But LSTMs didn't solve the core architectural problem. They still processed one token at a time, sequentially. And at massive scale, that was the killer.

Panel 2 — THE MEMORY BALL

CHAPTER 02

Three Problems ThatCrippled the Old Models

Before we get to the solution, let's be precise about the pain. The pre-Transformer era had three interconnected crises, and solvingany one of them would have been significant. The Transformer paper solved all three simultaneously.

⛓️

Problem 1: Sequential Bottleneck

RNNs and LSTMs process tokens one at a time. Step 2 cannot begin until Step 1 finishes. This means youcannot parallelize training across GPU cores. Training was agonisingly slow — weeks or months for large models.

🌫️

Problem 2: Vanishing Gradients

When you train a neural network with backpropagation, you compute gradients (error signals) and push them backwards through the chain. In a long sequence, those gradients shrink exponentially as they travel. Early tokens barely learn anything.

📏

Problem 3: Long-Range Amnesia

In the sentence"The trophy didn't fit in the suitcase becauseit was too big" — what does "it" refer to? The trophy. A human knows instantly. An RNN processing hundreds of words between "trophy" and "it" often forgot the connection entirely.

The Telephone Game at Scale

The vanishing gradient problem is best understood through the "telephone game" (Chinese Whispers). You whisper a sentence to the first person in a chain. By the time it reaches the 20th person, the message is garbled beyond recognition. In an RNN, the gradient signal is that whisper — and long sequences were destroying it.

LSTMs reduced the garbling with their gating mechanisms, but didn't eliminate it. And crucially, every single token in the sequence still had to wait for the one before it to finish processing. At a time when researchers were starting to dream about training on billions of words, this was a scaling cliff.

Panel 3 — THE VANISHING WHISPER

CHAPTER 03

A Spark Before the Fire —Bahdanau Attention (2014)

To be accurate about the history: the 2017 paper didn't invent attention from scratch. In 2014, Dzmitry Bahdanau and colleagues published a paper that added an "attention mechanism" on top of existing encoder-decoder RNNs for machine translation.

The idea was elegant: when generating each output word, instead of squishing the entire input sentence into one fixed-size vector, the model learns to "look back" at different parts of the input and assign weights — attention scores — to each input word. Generate "Hund" in German? Pay more attention to "dog" in the English source.

🏅 The 2014 Precursor

Bahdanau et al. (2014) showed attention worked. But they bolted iton top of RNNs — the sequential backbone was still there, just with a better look-back mechanism. It was like putting a turbocharged engine in a horse-drawn carriage.

The 2017 breakthrough came when Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin at Google Brain asked a radical question:what if we got rid of the carriage altogether?

"We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely."— Vaswani et al., Attention Is All You Need (2017)

CHAPTER 04

June 2017 —The Paper Drops

Eight Google Brain researchers quietly uploaded a pre-print to arXiv on June 12, 2017. The title was almost cheeky — "Attention Is All You Need" — a pun on The Beatles' "All You Need Is Love" and a direct challenge to the field:attention mechanisms alone are sufficient. No recurrence. No convolutions. Just attention.

The abstract was direct. They proposed the Transformer architecture, showed it achieved state-of-the-art on English-to-German translation (28.4 BLEU, surpassing the previous best by more than 2 BLEU points), trained it in a fraction of the time, and made a claim that would prove prophetic: this architecture was far more parallelisable and required significantly less time to train.

At the time, the machine learning community took notice but didn't immediately grasp the full magnitude. It looked like a better translation model. What it actually was, in retrospect: the foundation of every major AI system built in the next decade.

200,000+

Citations as of 2025 — one of the most cited papers in all of computer science history

Panel 4 — THE GOOGLE BRAIN MOMENT

CHAPTER 05

Self-Attention —Every Word Watches Every Word

This is the heart of it. Everything else in the Transformer paper is (brilliant) supporting machinery. Self-attention is the engine.

Here's the core idea in plain English:when processing any word, the model looks at every other word in the sentence simultaneously and calculates a relevance score. Instead of passing information along a chain one step at a time, every token "talks to" every other token in parallel.

The Library Analogy

Imagine a library where every book can send a little messenger to every other book, asking: "Hey, are you relevant to me?" Each pair of books gives an answer — a number from 0 to 1. The books with higher scores get to "share" more of their information when the library compiles its final report.

In the sentence"The cat sat on the mat because it was tired" — when the model processes "it", the self-attention mechanism computes a score between "it" and every other word. The word "cat" gets a very high score (because "it" refers to the cat), while words like "the" and "on" get low scores. This is done in one parallel operation — no sequential chain required.

The Math Behind It (Don't Panic)

The paper formalises this with three vectors derived from each word's embedding: aQuery (Q), aKey (K), and aValue (V). Think of it like a search engine:

🔍 Q / K / V Intuition

Query (Q): "What am I looking for?" — what the current word wants to know.
Key (K): "What do I offer?" — what each other word has to advertise.
Value (V): "Here's my actual content" — what each word shares if chosen.

Attention score =softmax( Q · Kᵀ / √d_k ) · V

The division by √d_k (square root of the dimension) is a stabilising trick — without it, the dot products can get very large and the softmax function becomes extremely "peaky" (everything goes to one word), which hurts training. The softmax then converts raw scores into a probability distribution — so all the weights add up to 1.0.

Self-Attention Weight Matrix — "it" attending to other words

The

cat

sat

on

mat

it

"it" →

0.05

0.72

0.04

0.03

0.10

0.06

"it" pays most attention to "cat" (0.72 weight) — this is how the model resolves co-reference. Higher = brighter.

The critical breakthrough isn't justthat these scores are computed — it'show:all pairs are computed in parallel using matrix multiplication. A sentence of 512 tokens doesn't require 512 sequential steps. It requires one big matrix operation that modern GPUs execute extremely fast. This is the parallelisation breakthrough that made scaling possible.

Panel 5 — WORDS LOOKING AT WORDS

CHAPTER 06

Multi-Head Attention —Many Perspectives at Once

Here's where the paper goes from clever to ingenious. A single self-attention computation gives you one view of how words relate. But language is rich — words relate to each other in manydifferent ways simultaneously.

Consider the sentence"She gave him the book she wrote":

— "she" and "him" have agrammatical subject/object relationship
— "she" (first occurrence) and "she" (second) have aco-reference relationship
— "book" and "wrote" have asemantic relationship (you write books)
— "gave" and "book" have averb-object relationship

One attention head would have to pick one of these.Multi-head attention runs several attention computations in parallel, each in a different "subspace" of the representation. The results are then concatenated and projected back to the original dimension.

Head 1

Grammatical roles (subject, object, verb)

Head 2

Co-reference resolution ("she" = "she")

Head 3

Semantic relatedness (book ↔ wrote)

Head 4

Syntactic dependencies (verb-object)

The original Transformer used8 attention heads. Modern LLMs like GPT-3 use 96, and models like Claude use even more. Each head develops its own specialisation during training — not by design, but emergently, because the model learns that different heads can capture different useful patterns.

🎭 The Right Analogy

Multi-head attention is like having a team of editors review your essay simultaneously. One editor focuses on grammar, one on logical flow, one on vocabulary, one on argument structure. You get all their feedback at once, then synthesise it. No editor has to wait for the previous one to finish.

Panel 6 — THE EIGHT-EYED TEAM

CHAPTER 07

Positional Encoding —Teaching Order Without Recurrence

Here's a subtle but critical problem. In an RNN, word order is implicit — you literally process word 1, then word 2, then word 3. The order is baked into the architecture. But in a Transformer, all words are processed in parallel. If you showed it "Dog bites man" and "Man bites dog" simultaneously, the attention mechanism alone would see the same set of words and might produce the same result.

That's obviously catastrophic for language. "The bank by the river" and "river by the bank the" mean very different things.

The solution:Positional Encoding. Before feeding word embeddings into the Transformer, you add a unique positional signal to each one. The paper uses a clever combination of sine and cosine functions at different frequencies:

📐 The Formula

PE(pos, 2i) = sin(pos / 10000^(2i/d))
PE(pos, 2i+1) = cos(pos / 10000^(2i/d))

Wherepos is the word's position andi is the dimension. The result: each position gets a unique, smooth vector that the model can learn to interpret. The sine/cosine waves at different frequencies are like a musical chord unique to each seat in the stadium.

Why sinusoids and not just the number 1, 2, 3...? Because sinusoids generalise. They allow the model to learnrelative positions (word 2 is one step after word 1) not just absolute ones. And they handle sequences longer than those seen in training gracefully, because the wave patterns extend naturally.

Modern variants like RoPE (Rotary Position Embedding, used in Llama and GPT-NeoX) and ALiBi have since improved on the original scheme — but they're all descendants of this 2017 insight.

Panel 7 — THE NUMBERED LINEUP

CHAPTER 08

The Full Transformer —Encoder + Decoder

The original paper was designed for sequence-to-sequence tasks — specifically machine translation. The architecture has two halves that work together: anEncoder that reads the input (the English sentence) and aDecoder that generates the output (the German translation).

The Encoder

The encoder stack (6 identical layers in the original paper) processes the entire input sentence in parallel. Each layer has two sub-components:(1) multi-head self-attention (all words attend to all other words) and(2) a feed-forward neural network applied to each position independently. Both sub-components useresidual connections (the input is added back to the output) andlayer normalisation — both stability tricks borrowed from computer vision.

The Decoder

The decoder is similar but has three sub-components per layer. The first ismasked self-attention — like encoder self-attention, but masked so that when generating word N, the model can only attend to words 1 through N-1 (it can't cheat by looking at future words). The second iscross-attention — the decoder attends to the encoder's output, connecting the input sentence to the generation process. The third is the same feed-forward network as in the encoder.

🧬 The Encoder-Decoder Legacy

BERT (2018) usesonly the encoder — great for understanding tasks (classification, named entity recognition). GPT-1/2/3/4 useonly the decoder — great for generation tasks (writing, code, conversation). The full encoder-decoder design lives on in models like T5 and BART, used heavily for translation and summarisation.

One more key ingredient:Feed-Forward layers. After each attention block, every position's representation passes through a small, identical 2-layer neural network. In the original paper, the inner dimension of this network was 2048 — 4× the model's embedding dimension of 512. In GPT-3, it's 4× 12,288 = 49,152. These layers are believed to act as "fact storage" — where knowledge learned during training gets encoded.

Panel 8 — THE TRANSFORMER MECHA

CHAPTER 09

The ImpactTimeline — 2017 to Now

The Transformer paper wasn't just a research curiosity. It was a platform. Within a year, the entire field had pivoted. Within five years, it had generated a trillion-dollar industry. Here's the direct lineage:

2017

"Attention Is All You Need"ORIGIN

Vaswani et al. (Google Brain). The original Transformer architecture. State-of-the-art on WMT English→German translation. 65M parameters. Training: 3.5 days on 8 P100 GPUs.

2018

BERT — GoogleENCODER-ONLY

Bidirectional Encoder Representations from Transformers. 340M parameters. Pre-trained on masked language modelling — predict randomly hidden words. Demolished 11 NLP benchmarks on release. The model that proved "pre-train, then fine-tune" as the dominant paradigm.

2018

GPT-1 — OpenAIDECODER-ONLY

Generative Pre-trained Transformer. 117M parameters. The first proof that a decoder-only Transformer, trained on unsupervised language modelling, could be fine-tuned for diverse tasks. OpenAI's foundational bet on the decoder path.

2019–2020

GPT-2 → GPT-3SCALE

GPT-2 (1.5B params) was so good at generating text that OpenAI staged a "staged release" over safety concerns. GPT-3 (175B params) — the first model to demonstrate serious few-shot learning. You could give it 3 examples and it could do a new task without any fine-tuning. The scaling laws paper (Kaplan et al.) proved bigger = better predictably.

2022

ChatGPT — RLHF Changes Everything

GPT-3.5 fine-tuned with Reinforcement Learning from Human Feedback (RLHF). The public suddenly had a conversational interface to an LLM. 1 million users in 5 days. 100M in 2 months. Fastest product growth in history. Every major tech company scrambled.

2023–2026

Claude · Grok · Gemini · Llama · Mistral · GPT-4/4o

All Transformer-based. All descendants of the 2017 paper. Claude (Anthropic) adds Constitutional AI. Llama (Meta) brings open-source to the frontier. Gemini (Google) goes multimodal. Grok (xAI) takes on real-time search. The Cambrian explosion of LLMs — every one of them tracing its lineage back to June 12, 2017.

Panel 9 — THE ROCKET OF PROGRESS

CHAPTER 10

Is It The Most Important AI PaperEver Written?

Let's be honest about this. The question is fascinating precisely because it's not entirely settled — there are serious candidates.

The Case For: Yes, Unambiguously

No single paper has had a more direct and immediate commercial and scientific impact in the modern AI era. Every frontier LLM in existence today is a Transformer. The trillion-dollar AI industry of the mid-2020s is built on this foundation. With 200,000+ citations, it's a runaway leader in citation counts for an ML paper. The research it unlocked — in text, image (ViT), audio (Whisper), protein structure (AlphaFold 2), video (Sora), code (Copilot) — spans essentially every domain of AI.

The Case For Other Contenders

🏆 Other Papers That Matter

Backpropagation (Rumelhart et al., 1986) — Without the ability to train neural networks at all, there's nothing to build on.

ImageNet + AlexNet (Krizhevsky et al., 2012) — The moment deep learning proved itself to the world, launching the modern deep learning era.

Word2Vec (Mikolov et al., 2013) — Showed that word embeddings encode semantic meaning; a prerequisite for Transformer input representations.

Scaling Laws (Kaplan et al., 2020) — Proved that LLM capabilities grow predictably with compute and data, enabling the investment thesis behind GPT-3 and everything after.

RLHF (Christiano et al., 2017) — The alignment technique that turned raw LLMs into assistants humans actually want to use.

The honest verdict:In the specific context of modern generative AI — LLMs, multimodal models, and the AI products billions of people use daily — "Attention Is All You Need" is the clearest single point of origin. Without backprop it couldn't exist, but without this paper, it wouldn't have become what it is. It's the right answer to the question "which paper made today's AI possible?"

"We are all standing on the shoulders of eight people who asked: what if recurrence isn't actually necessary?"— A reasonable paraphrase of the entire modern AI research community

Panel 10 — THE AI FAMILY PORTRAIT

CHAPTER 11

Where Do We Go FromHere?

The Transformer is dominant but not invincible. Researchers are actively working on what comes next — and several serious challengers are emerging.

The Current Limitations

Self-attention has a quadratic complexity problem. If your sequence has N tokens, the attention matrix is N × N. Double the sequence length, quadruple the compute. For long documents — books, codebases, hours of audio — this becomes brutally expensive. The context window you experience in Claude or GPT-4 represents enormous engineering effort to extend what was originally a very limited range.

What's Being Explored

Mamba / State Space Models (SSMs) process sequences in linear time, not quadratic — a genuine architectural alternative that some researchers believe could eventually rival or exceed Transformers for long-context tasks.Flash Attention (Dao et al., 2022) is an algorithmic optimisation that makes standard attention dramatically more memory-efficient without changing the math.Mixture of Experts (MoE) architectures — used in GPT-4 and Gemini — activate only a subset of parameters per token, allowing models with trillions of total parameters to run at the cost of a much smaller model.

Multimodality is the frontier. The Transformer's attention mechanism generalises naturally to images (patch tokens), audio (spectrogram tokens), video (frame tokens), and structured data. A single Transformer can in principle process all of these simultaneously — and models like GPT-4o and Gemini Ultra are moving rapidly in this direction.

The question researchers are now asking: is intelligence primarily a function of architecture, or of scale and data? The scaling laws suggest it's mostly the latter. If that's true, the Transformer need not be dethroned — it just needs to be fed more.

🔭 The Bigger Picture

We are, by most accounts, somewhere in the middle of the most important technological transition in human history. Every model at the frontier — the ones writing code, passing medical exams, generating video — shares a common ancestor. Eight researchers. One arXiv pre-print. Twelve hundred lines of Python. June 12, 2017.

Panel 11 — THE NEURAL SKY

CONCLUSION

The Seven Words ThatChanged Everything

We started this piece in 2016, watching RNNs and LSTMs struggle through their sequential chains, watching gradient signals vanish like whispers in a long corridor. We watched researchers work incredibly hard to coax these architectures to handle longer contexts, more complex language, bigger training sets — and hit wall after wall.

Then eight people asked a simple question —what if you just... paid attention? — and rewired the entire field.

The Transformer is not magic. It's mathematics: query-key-value lookups, scaled dot-product attention, layer normalisation, residual connections, feed-forward networks. Every piece is graspable. The genius was in the combination — and in the willingness to abandon the assumption that sequences must be processed sequentially.

The models you interact with today — the ones that draft your emails, explain your code, answer your questions about transformer architecture with exhaustive detail — are all running on this foundation. Claude (the AI that helped outline this post) is a Transformer. GPT-4 is a Transformer. Grok is a Transformer. Gemini is a Transformer. They are, in the deepest technical sense, all direct descendants of that arXiv upload.

Understanding "Attention Is All You Need" is not just historical curiosity. It's the grammar of modern AI. Once you understand it, you have a lens through which almost everything in the field makes sense — the scaling laws, the context window debates, the encoder vs. decoder architecture choices, the multimodal experiments, the efficiency research.

The paper is free. The arXiv link still works. It's 15 pages and reads more clearly than most ML papers. If this post piqued your curiosity: go read it. You'll understand it now.

2017 → ∞

The year a single paper changed the trajectory of intelligence itself

]]>Postsartificial-intelligencearchitectureautomationKnowledge Basehttps://curiousbit.netlify.app/building-with-llms-in-2026-the-framework-atlas/https://curiousbit.netlify.app/building-with-llms-in-2026-the-framework-atlas/Fri, 08 May 2026 00:00:00 +0000Ajay Walia<p>Four years after LLMs entered the mainstream, the single most common mistake I see architects make is spending most of their decision energy on the model. Which frontier model? GPT or Claude or Gemini? The model choice matters — but it is one decision out of roughly fifteen, and it is far from the most consequential one.</p><![CDATA[

Four years after LLMs entered the mainstream, the single most common mistake I see architects make is spending most of their decision energy on the model. Which frontier model? GPT or Claude or Gemini? The model choice matters — but it is one decision out of roughly fifteen, and it is far from the most consequential one.

Building an LLM-powered system in 2026 is an architecture decision made across a stack of competing frameworks, each solving a well-posed problem at a specific layer. I spent several months mapping that landscape as a practitioner — the result is a 73-page whitepaper I call the Framework Atlas. This post distils it into the five things I think every architect, engineer, and senior IT leader should know before picking a single tool.

---

The Stack Has a Shape

There is no single AI stack, but there is a canonical shape. Every non-trivial production LLM system — whether a support chatbot, a document search engine, or a multi-agent workflow — is a composition of six layers:

Application layer. The surface your user interacts with. LangChain is the default; Semantic Kernel is the Microsoft-native choice; CrewAI leads when the app itself is agentic.

Agent layer. When a single LLM call is not enough — when the system needs to plan, call tools, or coordinate among multiple agents — this layer provides the loop. LangGraph is the most production-grounded option in 2026.

Data / retrieval layer. The memory of your system. LlamaIndex leads on orchestration; Weaviate, Pinecone, and Chroma compete at storage, each tuned for a different operational profile.

Model layer. The foundation models themselves. This layer is increasingly commoditised. The most important design decision here is not which model you start with — it is whether you can swap it without rewriting the layers above.

Serving / inference layer. How you turn a model into an endpoint. vLLM dominates throughput-bound workloads; BentoML packages models into clean APIs for teams that want to think about models, not infrastructure.

Infrastructure layer. Kubernetes, Docker, cloud, on-prem. Every framework choice depends on where you can actually deploy.

Wrapped around all six layers are three concerns that have become non-negotiable since 2024:observability and evaluation,fine-tuning and training, andguardrails and safety. If your system design has no answer for any of these three, it is under-designed. Ignoring them does not eliminate the risk — it just defers it until something breaks in production.

---

The Abstraction Trap

Every framework in the atlas is catalogued against eleven attributes, but the one architects under-weight most consistently isabstraction level — how much code you write versus how much the framework decides for you.

LangChain’s high abstraction makes the first demo fast and the tenth production fix slow, because you are debugging through someone else’s default decisions. FAISS’s low abstraction costs more lines but yields fewer surprises at 3am.

The operational signal:match abstraction to team seniority. Junior teams over-value high abstraction; senior teams over-value low. A mixed team benefits from a medium default — and from making the choice explicitly rather than defaulting to whatever has the best GitHub star count.

---

Decision Heuristics That Actually Hold

Rather than optimising at each layer independently, the atlas maps common requirements to preferred framework combinations. These are the ones I have found most durable in practice:

Requirement	Starting stack
Fast LLM prototype	LangChain + Chroma + OpenAI API
Enterprise-grade RAG	LlamaIndex + Weaviate + LangSmith
Multi-agent workflow	LangGraph (+ AutoGen for agent conversations)
High-throughput inference	vLLM + Ray Serve
Local / offline / on-prem AI	Ollama + FAISS + LangChain-local
Domain-specialised model	Axolotl (QLoRA) + vLLM + MLflow

Two things stand out from this table. First, LangChain and LlamaIndex are not competitors — they compose cleanly, with LangChain at the application layer and LlamaIndex at the retrieval layer. Second, local inference is no longer an edge case. Ollama plus a Llama-3-class model is a realistic production option for regulated industries where data sovereignty is a hard constraint.

---

Agents Moved to Production — With Guardrails

In 2023, autonomous agents were mostly demos. By 2026, they are in targeted production use: triage, routing, research synthesis. What changed is not the models — it is the frameworks.

LangGraph’s state-machine model gives agents deterministic control flow: you declare states, transitions, and retry policies explicitly. AutoGen models multi-agent systems as conversations, which makes it remarkably expressive for critique-revise loops and planner-executor separations. The practitioner heuristic:for production agents, LangGraph. For multi-agent conversations, AutoGen. For lightweight document workflows, CrewAI.

The critical note:never deploy autonomous agents in production unless the failure cost is bounded. The agent should draft; a human should approve. The pattern that ships is almost always a hybrid — autonomy where the stakes are low, escalation where they are not.

Guardrails have crossed from afterthought to critical infrastructure in the same period. Prompt injection is the new SQL injection. Every production system needs an input guard, an output guard, and a policy layer between them. The minimum viable defense in 2026 is: input guard → LLM → output guard. Anything less is operating without a seat belt.

---

The 2026 Outlook: Three Trends Worth Designing For

Agents are becoming the compile target. LangGraph, AutoGen, and CrewAI are converging on a common abstraction — a loop over an LLM with tool use and state. Expect a future that looks like the deep learning layer in 2018: multiple frontend frameworks, one common runtime. Design your agent layer to be swappable.

Retrieval is eating search. Elasticsearch, Postgres, and OpenSearch all ship vector indexes now; Weaviate and Pinecone ship BM25. The primitives have converged. The differentiator is no longer features — it is operational maturity and the team’s ability to run the infrastructure. Hybrid retrieval (vector + keyword) is the production-safe default.

Guardrails are becoming infrastructure. Today they are a library you bolt on. In two years they will be a runtime — prompt injection detection, PII scrubbing, and policy enforcement applied by default to every model invocation, the way CORS and auth middleware is applied to every HTTP request today. Get ahead of this by treating your guardrails layer as critical infrastructure now, not as a compliance checkbox later.

---

A Practitioner’s Closing Note

Frameworks age faster than architectures. The stack shape you design today — application, agent, retrieval, model, serving, infrastructure — will still be valid in three years. The individual framework boxes you fill it with probably will not be. The single most important design invariant isswappability at each layer. Make the layer interfaces clean, keep the framework-specific code thin, and you will be able to move when the landscape shifts — and it will.

The full Framework Atlas (v4.0, April 2026) covers all ten framework categories in detail, including comparison tables, maturity radars, cost and latency envelopes, and four reference architectures with working code. It is available below.

Download the Framework Atlas — Building with LLMs v4.0 (PDF)

]]>Postsartificial-intelligencearchitecturellmframeworksengineeringKnowledge Basehttps://curiousbit.netlify.app/i-built-my-own-screenshot-app-for-macos-no-more-clunky-screenshots/https://curiousbit.netlify.app/i-built-my-own-screenshot-app-for-macos-no-more-clunky-screenshots/Fri, 08 May 2026 00:00:00 +0000Ajay Walia<p>It was a Tuesday evening. I had just taken my fourteenth screenshot of the day — a mix of Cmd+Shift+4, accidental desktop saves, files named <code>Screenshot 2026-05-06 at 11.43.22 PM.png</code> scattered across my Downloads folder like confetti after a party nobody enjoyed.</p><![CDATA[

It was a Tuesday evening. I had just taken my fourteenth screenshot of the day — a mix of Cmd+Shift+4, accidental desktop saves, files namedScreenshot 2026-05-06 at 11.43.22 PM.png scattered across my Downloads folder like confetti after a party nobody enjoyed.

I opened Finder. Forty-seven PNGs. Forty-seven. Some blurry. Some with my other windows bleeding in at the edges. Some cropped wrong because I’d sneezed mid-drag.

There had to be a better way.

---

The Ghost of Greenshot Past

Back in my Windows days — and I spent alot of years in Windows, running datacentres, managing Wintel estates, building VDI platforms — I hadGreenshot. If you’ve never used it, Greenshot is a free, lightweight screenshot tool that lives in your system tray. PressPrtSc, draw a box, done. Instant annotation. Instant clipboard. Instant sanity.

When I moved to macOS full-time, I expected something better. Apple builds beautiful hardware. Their software is generally excellent. And their screenshot workflow is… fine. Technically fine. But when you’re taking screenshots all day for documentation, Slack messages, architecture diagrams, and blog posts — “fine” isn’t good enough.

The native flow is:Cmd+Shift+4, drag imprecisely, file saves somewhere, you hunt for it, you paste it, you realise the crop was off, you do it again.

I Googled “Greenshot for Mac” approximately eleven times over the past year. The answers: CleanShot X (paid, subscription), Shottr (good but someone else’s decisions), or just “get used to it.”

I couldn’t get used to it. So I did what any reasonable enterprise architect does at 10pm on a Tuesday.

I decided to build it myself.

---

Fine. I’ll Build It.

The plan was simple: a lightweight macOS menu bar app. Lives in the status bar. One click or a hotkey and you’ve got a clean area capture, a window capture, or a full-screen grab. Auto-saves with a sensible filename. Copies to clipboard. Makes a satisfying shutter sound. Stays out of your way.

The twist: I’d build it the same way I built this blog — with Claude as my pair programmer and VS Code as the editor. I’m an architect and technologist, not a Swift developer. But that was the whole point.

I opened a new conversation, described what I wanted in plain English, and we got to work.

---

What AJShot Is

AJShot is a native macOS menu bar app — background-only, no Dock icon, no bloat. It sits quietly in your menu bar with a smallAJ badge. Left-click opens your last screenshot. Right-click opens the action menu.

Memory footprint: around50 MB at idle. Compare that to some Electron-based tools that idle at 300 MB just saying hello.

The right-click menu looks like this:

📷 Capture Area ⌘⇧3
🖥 Capture Window ⌘⇧4
🖥 Capture Fullscreen ⌘⇧5
─────────────────────────────
📂 Open Last Screenshot
📁 Open Screenshots Folder
─────────────────────────────
⚙️ Preferences
ℹ️ About
✕ Quit

The keyboard shortcuts are global — they work even when AJShot is in the background. You’re deep in a Zoom call, you need a quick capture, you hitCmd+Shift+3, draw the box, done. The screenshot is already in your designated folder and on your clipboard before you’ve even let go of the mouse.

---

Core Features

Capture Modes

Area Capture (⌘⇧3) — the workhorse. Full-screen overlay appears, cursor becomes a crosshair, you draw a box. Precise. Consistent. No more dragging the wrong direction and getting a sliver of your taskbar.

Window Capture (⌘⇧4) — click a window, capture just that window. No background bleed, no accidental desktop icons photobombing your documentation.

Fullscreen Capture (⌘⇧5) — captures everything. Supports multi-display setups with a “capture all” or “ask each time” behaviour you configure once.

Preferences That Actually Make Sense

Open Preferences and you get a clean panel with the options you’d actually want to configure:

Setting	Default	What it does
Launch at login	On	AJShot is always ready, even after restart
Play capture sound	On	Satisfying shutter click confirms the capture
Show thumbnail preview	On	Floating preview appears — click to open in editor
Auto-copy to clipboard	On	Screenshot lands in your clipboard automatically
Save folder	~/Pictures/AJShot	Your own designated screenshot home
File format	PNG	Or JPG with a quality slider
Filename template	AJShot_{date}_{time}	Consistent, sortable, sane filenames

The filename templating engine handles the sanitisation itself — no illegal characters, no trailing dots, no 240-character filenames. Files come out likeAJShot_2026-05-09_22-14-37.png. You can find them instantly. You can sort them. Your Downloads folder stays clean.

Post-Capture Flow

After every capture, AJShot runs through a quick sequence:

1. Auto-saves to your configured folder with the filename template applied
2. Copies to clipboard (if enabled) — so you can paste immediately
3. Plays the shutter sound — audio confirmation that the capture happened
4. Shows a floating thumbnail in the corner — click it to open in the editor

You can also configure the post-capture action to always ask, always edit, always save, or always copy. Once you’ve decided how you work, it remembers.

---

The Real Problem It Solves

Here’s the honest comparison:

	Native macOS	AJShot
Start capture	Cmd+Shift+4, wait for toolbar	Cmd+Shift+3, instant overlay
File location	Desktop or last used folder	Always~/Pictures/AJShot
Filename	Screenshot 2026-05-09 at 10.14.22 PM.png	AJShot_2026-05-09_22-14-22.png
Clipboard copy	Manual extra step	Automatic
Editor	Preview.app detour	Built-in (annotation tools in progress)
Memory use	System process	~50 MB dedicated
Works in background	Partial	Full global hotkeys
Shutter sound	Yes	Yes (configurable)

The editor is where things get even more interesting. The annotation scaffolding is already built — the tools are there in the codebase: arrows, blur, text, highlights, shapes, step-number callouts. They’re not wired to the UI yet, but they’re coming. That’s the next milestone.

---

Tech Stack — What’s Actually Under the Hood

This is anative macOS app built entirely inSwift 5.9+, targetingmacOS 12 (Monterey) and later. The UI is a hybrid ofSwiftUI andAppKit — SwiftUI for the Preferences panel and modern views, AppKit where you need fine-grained control over system behaviour (status bar items, window management, capture overlays).

Dependencies (both bySindre Sorhus, the prolific open-source developer behind half the macOS indie tool ecosystem):

• KeyboardShortcuts — global hotkey registration that plays nicely with macOS security
• Defaults — a type-safe, SwiftUI-friendly wrapper aroundUserDefaults

Capture pipeline: usesCGPreflightScreenCaptureAccess andCGRequestScreenCaptureAccess from CoreGraphics for the permission preflight, thenScreenCaptureKit for the actual pixel capture. The permission flow has a proper retry loop — if you’ve granted permission but haven’t restarted the app, it tells you exactly that and offers a one-click restart.

Build and distribution: Swift Package Manager for dependencies, Xcode 15 for the build, and abuild-dmg.sh script that produces a signedAJShot-1.0.0.dmg for distribution. The DMG is already built. You can drag it to Applications and run it today.

Security considerations (because I work in security and these things matter):

• Screenshot folder is set to0700 permissions — only you can read it
• Individual screenshot files are0600 — same
• Filename template engine strips illegal characters and control characters at the source
• Code signing and notarization stubs are already in the README for when distribution goes wide

The architecture is clean:App →AppDelegate →StatusBarController →CaptureManager →ScreenshotStorage. Each module does one thing. TheFilenameTemplateEngine is a pure static function. TheThumbnailPresenter is decoupled from capture. Claude helped me keep it disciplined.

---

The Honest Struggles

No developer story is complete without the bit where things go sideways.

Screen Recording permissions nearly broke me on the first build. macOS 12+ requires explicit Screen Recording permission in System Settings, and you can’t force-trigger the dialog more than once per session. I had to build the multi-stage fallback: first launch triggers Apple’s native dialog, subsequent denials open System Settings directly, and if you’ve granted permission but not restarted, it detects that state and offers a restart button. It took longer to get that right than the actual capture code.

Swift Package Manager vs Xcode had a brief disagreement about the resource bundle forshutter.aiff. The sound file lives inAJShot/Sounds/ and has to be declared as a.process("Sounds") resource inPackage.swift. Simple when you know it. Less simple at 11pm when it just silently fails to play.

TheLaunchAtLogin manager required writing aLaunchAgent plist to~/Library/LaunchAgents/. Straightforward on paper. In practice, macOS is protective of that folder in ways that aren’t well-documented, and the error messages when something goes wrong are the kind that send you to Stack Overflow threads from 2019.

That’s the thing about building a native macOS app — the platform is powerful and the APIs are solid, but the surface area of “things that can quietly not work” is larger than you’d expect. Claude helped me navigate most of it. The frustration was real and so was the progress.

---

What’s Next

The annotation editor is the next major milestone — the scaffolding is already there. Arrow tool, blur tool, text tool, highlight, shapes, step-number callouts. When that’s wired up, AJShot becomes a complete Greenshot replacement for macOS: capture, annotate, save, share.

After that: GitHub release with a public DMG, code signing, notarization, and probably a product page here on CuriousBit.

If you’re a macOS user who’s been putting up with the native screenshot workflow out of habit — you don’t have to. And if you’re a developer (or aspiring one) who thinks you can’t build a native app without being a full-time Swift engineer — you can. You really can.

---

Try It / Share What You Use

The GitHub repo is coming soon — I’ll post the link here when it’s public. In the meantime, if you’re curious about the architecture, the Swift source, or want to follow along as the editor gets built, keep an eye on this blog.

And I’m genuinely curious:what screenshot tool do you use on macOS? Are you a CleanShot loyalist? A Shottr person? Still using the native tools and somehow thriving? Drop a note — I’d love to know.

Because the best tools are the ones built out of genuine frustration with the alternative. And I wasvery genuinely frustrated.

]]>Postsartificial-intelligenceautomationarchitectureKnowledge Basehttps://curiousbit.netlify.app/knowledge-distillation-from-massive-models-to-efficient-intelligence/https://curiousbit.netlify.app/knowledge-distillation-from-massive-models-to-efficient-intelligence/Fri, 08 May 2026 00:00:00 +0000Ajay Walia<p>There is a scene you have probably seen in countless films: a master craftsman, decades of experience locked in his hands, patiently guiding a young apprentice. The master does not hand over a textbook. He transfers something richer — intuition, nuance, an understanding of <em>why</em> certain choices matter. The apprentice, unburdened by the master&rsquo;s size and slowness, eventually moves faster and in some cases surpasses the teacher entirely.</p><![CDATA[

There is a scene you have probably seen in countless films: a master craftsman, decades of experience locked in his hands, patiently guiding a young apprentice. The master does not hand over a textbook. He transfers something richer — intuition, nuance, an understanding ofwhy certain choices matter. The apprentice, unburdened by the master’s size and slowness, eventually moves faster and in some cases surpasses the teacher entirely.

Knowledge Distillation is that scene, rendered in mathematics.

Introduced formally by Geoffrey Hinton, Oriol Vinyals, and Jeff Dean at Google in 2015, Knowledge Distillation (KD) is a model compression technique where a large, expensive model — theteacher — transfers its learned intelligence to a compact, deployable model — thestudent. The student retains over 90% of the teacher’s accuracy while being up to 100× smaller and faster.

This article takes you from the intuition all the way through to the advanced variants that are reshaping AI deployment in 2026.

---

The Problem: Intelligence Is Expensive

Modern AI models are enormous. GPT-4 is estimated to contain over a trillion parameters. BERT-large has 340 million. These models achieve stunning accuracy — but they are cumbersome to deploy. Running a trillion-parameter model for every user query would require data centres the size of small cities.

The engineering instinct is to train a smaller model directly. But smaller models trained from scratch on raw data consistently underperform large ones. Why?

Because raw training data ishard. A cat photo labelled simply “cat” gives a small model very little to work with. A large model, however, does not just see “cat” — it sees a distribution of confidence across thousands of classes. “Cat: 0.92, Lynx: 0.06, Tabby: 0.02.” That probability distribution is enormously richer than the hard label.

Hinton called this richer signaldark knowledge — the information encoded in what the modelalmost predicted.

---

The Teacher-Student Paradigm

The core idea is elegant. Instead of training the student on raw labelled data, you train it tomimic the teacher’s output distribution.

You run every training example through the large teacher model. For each example, instead of a hard label (0 or 1), you collect the teacher’s fullsoft target — the probability it assigns to every possible class. You then train the student to produce those same soft probability distributions.

The student loss function becomes:

Loss = α × (cross-entropy with hard labels)
+ (1-α) × (KL divergence from teacher soft targets)

The blending weightα controls how much the student learns from the raw data versus the teacher’s guidance. In practice, a smallα (more weight on teacher targets) is usually optimal.

---

Soft Targets and Dark Knowledge

Hard labels are binary. Soft targets are continuous. That difference is enormous.

Consider an image of a dog that slightly resembles a wolf. A hard label says “dog: 1, wolf: 0.” A teacher that has seen millions of examples says “dog: 0.84, wolf: 0.13, fox: 0.03.” That residual probability onwolf carries genuine information about the visual ambiguity in the image. The student trained on soft targets learns not just the answer, but theshape of uncertainty around the answer.

This is the dark knowledge. It lives in the tails of the distribution — the non-zero probabilities on wrong answers — and it makes the student dramatically more robust than one trained on hard labels alone.

---

Temperature: The Control Knob

Soft targets, by default, tend to be very peaked — the teacher is often highly confident in its top prediction, assigning 0.99 to the correct class and tiny residuals to everything else. At that extreme, the soft target is barely different from a hard label, and the dark knowledge disappears.

Hinton’s solution wastemperature scaling. Before computing the softmax, you divide the logits by a temperature parameter T:

p_i = exp(z_i / T) / Σ exp(z_j / T)

AtT = 1 (standard), outputs are sharp and peaked. AtT > 1 (high temperature), outputs become softer and more spread, revealing the relative confidence structure across all classes.

During distillation, both teacher and student use the same elevated temperature (typically T = 3–5). This “warms up” the teacher’s output into a richer, more informative distribution for the student to learn from. After training, the student is deployed with T = 1.

The effect is striking. Higher temperatures expose more inter-class structure, giving the student a better map of the concept landscape rather than just a list of correct answers.

---

What Gets Transferred? Three Flavours of Distillation

Knowledge can flow from teacher to student in different ways. The research community has converged on three main categories:

Response-based distillation — the original Hinton approach. The student matches the teacher’s final output layer (soft targets). Simple, effective, widely used.

Feature-based distillation — the student is trained to match not just the final output but intermediate representations — specific layers or attention maps inside the teacher. This transfershow the teacher thinks, not just what it concludes. The trade-off is complexity: the teacher and student must have compatible architectures or an adapter layer is needed.

Relation-based distillation — the student learns to replicate therelationships between different training examples as the teacher sees them. If the teacher places cat images and dog images in nearby regions of its feature space, the student should too. This approach is particularly powerful for metric learning and few-shot tasks.

---

Advanced Variants

Multi-Task Distillation

Microsoft’s MT-DNN research showed that distillation composes naturally with multi-task learning. A teacher trained on nine different natural language tasks simultaneously was distilled into a single student model. The distilled MT-DNN outperformed the original on 7 of 9 GLUE benchmark tasks — pushing the single-model state of the art to 83.7%.

The insight: when a teacher has learned to generalise across many domains, its soft targets encode cross-task structure that a specialised student cannot discover on its own.

The Teacher Assistant Bridge

What happens when the teacher and student are so different in capacity that direct distillation fails? A very large teacher produces soft targets the tiny student simply cannot model well.

The solution is an intermediateTeacher Assistant (TA) — a medium-sized model that first distils from the large teacher, then acts as teacher to the small student. The TA bridges the capacity gap, giving the small student a more tractable target. Research has consistently shown this staged approach outperforms direct large-to-small distillation when the size gap is more than an order of magnitude.

When the Student Surpasses the Teacher

One of the most counter-intuitive findings in knowledge distillation is that the student can sometimesexceed the teacher.

The 2022Symbolic Knowledge Distillation paper demonstrated this dramatically. The researchers distilled commonsense reasoning from GPT-3 (175B parameters) into a purpose-built commonsense model at 100× smaller size. The resulting student — COMET-DISTIL — outperformed GPT-3 on commonsense benchmarks.

How? The distillation process acted as a filter. Rather than transferring all of GPT-3’s knowledge, the researchers used acritic model to selectively distil only high-quality, high-confidence commonsense triples. The student was not burdened by GPT-3’s off-topic knowledge or low-confidence noise. It received a curated, concentrated version of the teacher’s relevant expertise.

This is the Renaissance apprentice story made literal: the student, given the master’s best knowledge and freed from the master’s constraints, eventually does better work.

---

Real-World Results

The numbers behind knowledge distillation are worth anchoring:

In Hinton’s original speech recognition experiments on a heavily used commercial system, a distilled single modelmatched the accuracy of a 10-model ensemble while requiring one-tenth the compute at inference time.

In the speech recognition benchmark specifically:

Model	Frame Accuracy	Word Error Rate
Baseline (single model)	50.9%	10.9%
10× model ensemble (teacher)	61.1%	10.7%
Distilled single student	60.8%	10.7%

The student matches the ensemble at a fraction of the cost. This is the central promise of KD — and it has held up across vision, language, and speech for over a decade.

---

Why This Matters in 2026

Knowledge distillation is no longer a research technique. It is infrastructure.

Every major on-device AI model — the language models on your phone, the vision models in your camera, the wake-word detectors in your earbuds — was almost certainly distilled from a much larger cloud model. DistilBERT, MobileNet, and Whisper Tiny are all products of distillation.

The technique is also central to the LLM compression wave of the past two years. Models like Phi-3, Mistral Small, and Gemma were designed with distillation-aware training pipelines from the start. The goal: deliver GPT-4-class reasoning in a model small enough to run locally, privately, and cheaply.

And symbolic distillation — transferring knowledge as structured text rather than as neural activations — is opening entirely new territory, allowing language model intelligence to flow into specialised domain models that do not even share the same architecture.

---

A Practical Starting Point

If you want to experiment with knowledge distillation today:

For response-based KD in PyTorch, the training loop change is minimal — replace your standard cross-entropy loss with the blended loss described above and pass the teacher’s logits alongside the hard labels.

For NLP tasks, Hugging Face’stransformers library includes DistilBERT as a reference distilled model with its training recipe documented.

For vision, TorchVision’s knowledge distillation tutorial is the fastest on-ramp.

The key design decisions are: the temperature T (start at 4), the blending weight α (start at 0.5), and whether you need feature-based or response-based transfer (response-based first, feature-based if accuracy is still insufficient).

---

The master-apprentice metaphor is more than decorative. Knowledge distillation encodes a genuine pedagogical insight: that the richer the guidance a learner receives, the more efficiently it reaches competence. The hard labels of raw data are the equivalent of telling a student the answer. The soft targets of a teacher model are the equivalent of showing them how to think.

That distinction — answer versus thinking — is what makes knowledge distillation one of the most elegant ideas in modern machine learning.

]]>Postsartificial-intelligencemachine-learningmodel-compressiondeep-learningKnowledge Basehttps://curiousbit.netlify.app/i-built-this-blog-without-writing-a-single-line-of-code-almost/https://curiousbit.netlify.app/i-built-this-blog-without-writing-a-single-line-of-code-almost/Thu, 07 May 2026 00:00:00 +0000Ajay Walia<p>It started like most late-night ideas in Sydney — a MacBook, too much coffee, and a thought that refused to go away.</p> <p><img src="https://curiousbit.netlify.app/images/comic-p1.jpg" alt="Panel 1 – Sydney, 11:47 PM. The idea hits."></p><![CDATA[

It started like most late-night ideas in Sydney — a MacBook, too much coffee, and a thought that refused to go away.

I wanted a tech blog. A proper one — clean, fast, with a Knowledge Base, Videos section, and a place to put thoughts on AI and enterprise tech. The problem? Zero design skills. Zero frontend experience. I know infrastructure, architecture, platforms — but CSS gives me a headache.

---

The Crazy Idea

Then it clicked.

What if I just described the site I wanted — in plain English — and let AI build it? Not generate a snippet. Build thewhole thing. Layouts, config, content, deployment config. Everything.

The prompt I used was simple but specific:

“Make me a clean HackerNoon-style Hugo + Tailwind blog with KB, Videos, News & Views, and About pages. Deploy-ready for Netlify.”

---

The Stack

The AI returned a complete project in one shot. Here’s what it chose and why it makes sense:

Hugo — a static site generator written in Go. Blazing fast builds, no database, no server to maintain. Perfect for a content blog.

Tailwind CSS — utility-first CSS. Instead of writing stylesheets, you compose classes directly in HTML. The AI could reason about it well and generate clean, consistent UI.

Netlify — one-click deployment from a GitHub repo. Push tomain, site rebuilds automatically. Free tier covers everything a personal blog needs.

GitHub — version control and the bridge between local edits and live site.

The structure it generated:content/,layouts/,assets/,static/,config.yaml,netlify.toml. Exactly what you’d expect from an experienced Hugo developer.

---

First Roadblock — Hit in Under 5 Minutes

Real talk — the first thing I hit was a terminal error before I’d even installed anything.zsh: command not found: brew. Homebrew wasn’t on the machine.

At first I had no idea what I was looking at.

But here’s the thing about working with AI tools — you learn to debug faster because you can askwhy something failed, not justhow to fix it. Within a few minutes I understood: Homebrew is the Mac package manager. I needed it to install Hugo. Small win — I could read what was going wrong.

---

Getting Hugo Running

Three commands and Hugo was running:

/bin/bash -c"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"brew install hugohugo server

The site was alive atlocalhost:1313 — navigation, article cards, dark mode toggle, the works.

---

The Project Structure

The generated project was clean and logical:

• content/posts/ — articles as Markdown files with YAML frontmatter
• layouts/ — Hugo HTML templates for each page type
• assets/css/ — Tailwind CSS, processed at build time
• static/ — images and files served as-is
• config.yaml — site title, menus, author, base URL
• netlify.toml — build command, publish directory, Node version

Hugo reads all frontmatter at build time and generates a completely static site. No database. No server-side rendering. No WordPress. Just fast HTML files.

---

Real Developer Moments

Not everything was smooth. There were genuine “how does anyone actually do this” moments — forgetting how to copy a file path in Finder, templates not updating because of Hugo’s build cache, Git authentication failing because GitHub dropped password support in 2021.

These are the moments tutorials skip. The AI handled every one of them.

---

Deployed. It’s Alive.

Getting it live was surprisingly painless:

1. Push the project to a GitHub repo
2. Connect the repo to Netlify
3. Set build command tonpm run build, publish directory topublic/
4. Hit deploy

Netlify pulled the code, ran Hugo, and published the static site in under 30 seconds. Everygit push from that point triggers an automatic rebuild.

---

What the Live Site Looks Like

The site has:

• Homepage — featured article hero, rotating video cards, Knowledge Base grid
• KB — long-form technical articles in Markdown
• Videos — auto-synced from a YouTube playlist via YouTube Data API v3. A Node.js script fetches the playlist at build time, writesvideos.json tostatic/, and the browser loads it at runtime. Add a video to YouTube, trigger a deploy, it appears on the site.
• About — a simple bio

The video section was the most technically interesting piece. No manual uploads, no embeds to maintain — just a playlist that feeds itself.

---

The Takeaway

This entire site — layouts, templates, CSS, YouTube integration, Git workflow, deployment pipeline — was built by someone with no frontend background. What I brought was the ability to describe what I wanted clearly, debug errors methodically, and push through the friction points.

The tools did the heavy lifting. The judgment aboutwhat to build was still mine.

---

Now It’s Your Turn

The stack is:Hugo + Tailwind + GitHub + Netlify + Claude. All free. All production-grade.

Start with a prompt. Describe what you want clearly. Expect errors. Fix them one at a time. Deploy early and often.

The gap between “I have an idea” and “it’s live on the internet” has never been smaller.

---

The full source for this site is on GitHub atibn-Battuta/AjayW_blog.

]]>Postsartificial-intelligencearchitectureautomationKnowledge Basehttps://curiousbit.netlify.app/camera-roll-to-caption-python-pipeline-vision-model-for-photo-tags/https://curiousbit.netlify.app/camera-roll-to-caption-python-pipeline-vision-model-for-photo-tags/Sat, 02 May 2026 00:00:00 +0000Ajay Walia<p>Vision models, language models, and most other generative systems are confident-but-wrong some non-trivial fraction of the time. The instinct is to fix that with better prompts, bigger models, or smarter agents. The cheaper move is usually to add a small structured review seam — a thirty-second checkpoint where a human can glance, correct, and move on.</p><![CDATA[

Vision models, language models, and most other generative systems are confident-but-wrong some non-trivial fraction of the time. The instinct is to fix that with better prompts, bigger models, or smarter agents. The cheaper move is usually to add a small structured review seam — a thirty-second checkpoint where a human can glance, correct, and move on.

This post is the case study for one such seam, dropped into a build I needed for myself. Of 35 garden photos handed to a vision model,74% came back with correct first-pass labels. After thirty seconds editing a CSV,97% were acceptable to publish. Total API cost:$0.18. Total inference time:~74 seconds at 2.1 sec/photo ongpt-4o-mini. The CSV was the highest-leverage code in the project — and it isn’t really code.

Here’s the story.

The annoyance

It was a Saturday afternoon in early March. I’d come back from a walk around the garden with thirty-five photos on my iPhone — bottlebrush in full red, honeysuckle dripping with rain, a lilly-pilly cluster doing its outrageous pink thing, and at least one inexplicable shot of an old railway station I’d passed on the way home.

I wanted to post a handful of them with consistent little hashtag labels —#bottlebrush,#honeysuckle,#flower — burned into the corner like a quiet caption. Not a watermark, not a filter, just a small readable pill that says “this is what you’re looking at.”

What I didn’t want was to open each HEIC in Preview, draw a text box, fiddle with the font, export, repeat thirty-five times. So I did the only reasonable thing: I wrote a small Python tool that does it for me.

The shape of the pipeline

┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Folder │──▶│ Vision │──▶│ CSV │──▶│ Apply │──▶│ Tagged │
│ photos │ │ provider │ │ review │ │ + pill │ │ output │
└──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘
↑
human-in-the-loop seam
--mode propose: folder ─▶ vision provider ─▶ CSV
--mode apply: CSV ─▶ render + pill ─▶ tagged output

The minimal product was easy to describe. Point the script at a folder. For each image — HEIC, JPG, PNG, whatever the iPhone or my camera roll throws at it — open it, figure out what’s in it, draw a small rounded hashtag pill into the bottom-right corner, save the result to atagged_output/ subfolder. No watermark across the centre of the image, no filter or colour grade, no destructive edit to the original, and no making me choose the label by hand when a vision model can have a decent first guess.

That last point is where the design got interesting.

The seam

You could write this as a single command: walk the folder, ask the model, render the tag, done. I tried that first. The first-pass run produced a folder of beautifully tagged images, about a quarter of which were wrong in some quietly maddening way — a daisy called#flower, a fern called#leaves, the railway station called, charitably,#station.

So the script runs in two passes.

--mode propose opens each image, hands it to the vision model, and writes a CSV with five columns:

image_path, label, score, suggested_tag, final_tag

final_tag is initialised tosuggested_tag, but the whole point of the column is that you can edit it. Open the CSV, glance down the list, fix anything obvious —flower becomesmorning_glory,leaves becomesbamboo — save, close. On this batch,9 of 35 rows needed editing (a daisy, the railway station, two ferns, the bamboo, and four generic-flower fallbacks). A thirty-second pass.

--mode apply then reads the CSV row by row and renders the tag using whatever’s infinal_tag. The CSV is the human-in-the-loop seam. It is much cheaper than re-running inference, and it catches the cases where the model was right about the genus but wrong about the species, or just wrong.

Three providers, one interface

I didn’t want to commit to one vision model — the price/quality trade-offs are too lively right now. The script supports three providers behind one interface, picked via--provider local|openai|xai.

Local CLIP. HuggingFace’sopenai/clip-vit-large-patch14 against a fixed candidate list. Free, offline,~0.4 sec/photo on an M3 Pro. The cost is breadth: anything outside the candidate list collapses to the nearest match. CLIP doesn’t know what a bottlebrush is unless I tell it the word.

OpenAI.gpt-4o-mini by default, with an opt-in--high-accuracy flag that retries low-confidence cases (under 0.72) ongpt-4o.~2.1 sec/photo, ~$0.18 for the 35-photo batch. Open-ended labels — howbottlebrush,honeysuckle,fern, andberries ended up in the CSV rather thanflower,flower,leaves,fruit.22% of the batch tripped the retry threshold and went togpt-4o.

xAI Grok. Same OpenAI-compatible client, pointed atapi.x.ai withgrok-2-vision-latest. Useful if you’re already on the x.ai stack or want a different model family’s vote.

The mental model: local CLIP for batch-of-a-hundred-photos-on-a-flight, OpenAI as the daily driver, and the high-accuracy retry for exactly the case where the model says “flower” with 0.55 confidence and I want it to look harder before I have to.

The blue morning glory below is what generic labels look like in practice — still a decent fallback, just unspecific. The model wasn’t wrong; it just wasn’t curious.

Two small touches

Two design choices are the difference between “the script works” and “the output looks intentional.”

Style-aware contrast. The pill needs to be readable on both a bright sky and dark foliage. The script crops the bottom-right region of the image, measures the mean luminance using the standard Rec. 709 weights, and flips the colour scheme above or below a threshold:

defstyle_aware_colors(img):w,h=img.sizecrop=img.crop((int(w*0.68),int(h*0.80),w,h))r,g,b=ImageStat.Stat(crop.convert("RGB")).mean[:3]luminance=0.2126*r+0.7152*g+0.0722*bifluminance<140:return(255,255,255,245),(0,0,0,95)# white text, dark pillreturn(0,0,0,245),(255,255,255,95)# black text, light pill

Eight lines of PIL. In this batch every photo sampled dark — gardens are mostly green and shadow in the corner — so every output got the dark pill. The bright-pill branch is still there, waiting for a photo with sky or a light wall in the corner.

Save with fallback. HEIC writes occasionally fail for reasons that aren’t worth diagnosing in a personal tool. The save function tries the original format first; if PIL throws, it quietly drops to JPEG with the same filename stem. Eight more lines. On this batch,3 of 35 fell back to JPEG. Without the fallback those three would have been a stack trace and a half-finished folder. With it, thirty-five of thirty-five made it through.

What I’d add next

Multi-tag support, so a photo can be#lorikeet #bottlebrush when the bird showed up in the bottlebrush. EXIF preservation through the round-trip — right now PIL strips most of the metadata, which I don’t love. A tiny review UI to replace the CSV step, either a Tkinter window or a one-page localhost app. Smarter candidate lists for the local provider, scoped by season or geography — Sydney summer has a different vocabulary than European spring.

None of these are urgent enough to displace “the script already does what I wanted.”

Closing observations

Three lessons that generalise beyond this script.

Human-in-the-loop is cheap and underrated. The CSV seam between propose and apply takes thirty seconds per batch and saves me from confidently wrong outputs. For any task where a model is confident-but-wrong some non-trivial fraction of the time — RAG, codegen, moderation, enterprise copilots, agentic workflows — a structured review step pays for itself almost immediately. The CSV doesn’t have to be elegant. It has to exist.

Pluggable providers are worth the small abstraction tax even on personal tools. I went from local CLIP togpt-4o-mini to Grok in the space of one afternoon without rewriting the rendering code. The interface is(client, model, image) → (label, score) and that’s it. Once you’ve paid that cost once, you can keep up with a fast-moving model market essentially for free.

Small touches decide whether a script feels finished. Luminance-aware contrast and a save-format fallback don’t change what the tool does; they change how the output reads.

The model wasn’t the product. The seam was.

---

A short reel of the tagged photos in the wild:Instagram story.

]]>Postsautomationartificial-intelligencebuild-logengineeringKnowledge Basehttps://curiousbit.netlify.app/a-field-guide-to-ai-chips/https://curiousbit.netlify.app/a-field-guide-to-ai-chips/Wed, 29 Apr 2026 00:00:00 +0000Ajay Walia<style> @import url('https://fonts.googleapis.com/css2?family=Cinzel:wght@500;700;900&family=IM+Fell+English:ital@0;1&family=Inter:wght@400;500;600&display=swap'); .chip-guide { --bg: #0a1424; --surface: #14253f; --surface2: #1c2d4f; --border: #3a4a6a; --bronze: #b87333; --bronze-bright: #d4a04a; --gold: #e5b85e; --parchment: #e8e0d0; --muted: #9aa6bf; --accent-purple: #8b5cf6; --accent-teal: #2dd4bf; color: var(--parchment); font-family: 'Inter', sans-serif; font-size: 17px; line-height: 1.75; } .chip-guide h1, .chip-guide h2, .chip-guide h3 { font-family: 'Cinzel', serif; letter-spacing: 0.04em; color: var(--gold); font-weight: 700; margin-top: 2.5rem; margin-bottom: 1rem; } .chip-guide h1 { font-size: clamp(2.2rem, 4vw, 3.2rem); font-weight: 900; background: linear-gradient(135deg, #e5b85e 0%, #d4a04a 50%, #b87333 100%); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text; text-align: center; margin: 0.5rem 0 0.5rem; } .chip-guide h2 { font-size: clamp(1.4rem, 2.2vw, 2rem); border-bottom: 2px double var(--bronze); padding-bottom: 0.4rem; } .chip-guide h3 { font-size: clamp(1.2rem, 1.6vw, 1.5rem); color: var(--bronze-bright); } .chip-guide p { margin: 0.6rem 0 1rem; } .chip-guide strong { color: var(--gold); } .chip-guide a { color: var(--bronze-bright); } .chip-guide a:hover { color: var(--gold); } .chip-guide .hero { text-align: center; padding: 2.5rem 0 1.5rem; border-bottom: 1px solid var(--border); margin-bottom: 2rem; } .chip-guide .eyebrow { display: inline-block; font-family: 'Cinzel', serif; font-size: 0.78rem; letter-spacing: 0.35em; color: var(--bronze-bright); text-transform: uppercase; margin-bottom: 0.6rem; } .chip-guide .subtitle { font-family: 'IM Fell English', serif; font-style: italic; font-size: clamp(1.05rem, 1.4vw, 1.3rem); color: var(--muted); margin: 0.8rem auto 0; max-width: 720px; } .chip-guide .ornament { text-align: center; margin: 1.2rem 0; color: var(--bronze); font-size: 1.3rem; letter-spacing: 0.6rem; } .chip-guide table { width: 100%; border-collapse: collapse; margin: 1.5rem 0; font-size: 0.95rem; background: var(--surface); border: 1px solid var(--border); } .chip-guide table th { background: var(--surface2); color: var(--gold); font-family: 'Cinzel', serif; text-align: left; padding: 0.7rem 1rem; border-bottom: 2px solid var(--bronze); font-weight: 600; letter-spacing: 0.04em; font-size: 0.92rem; } .chip-guide table td { padding: 0.7rem 1rem; border-bottom: 1px solid var(--border); vertical-align: top; color: var(--parchment); } .chip-guide table tr:last-child td { border-bottom: none; } .chip-guide table tr:hover td { background: rgba(229, 184, 94, 0.04); } .chip-guide .stat-block { background: var(--surface); border: 2px solid var(--bronze); border-radius: 4px; margin: 1.5rem 0; font-family: 'Inter', sans-serif; } .chip-guide .stat-block-header { padding: 0.55rem 1rem; background: var(--surface2); border-bottom: 2px solid var(--bronze); font-family: 'Cinzel', serif; font-weight: 700; color: var(--gold); letter-spacing: 0.05em; font-size: 1.05rem; text-transform: uppercase; } .chip-guide .stat-block dl { padding: 0.7rem 1rem 0.8rem; margin: 0; display: grid; grid-template-columns: 8rem 1fr; row-gap: 0.4rem; column-gap: 0.6rem; } .chip-guide .stat-block dt { font-weight: 700; color: var(--bronze-bright); font-family: 'Cinzel', serif; font-size: 0.82rem; letter-spacing: 0.05em; text-transform: uppercase; margin: 0; } .chip-guide .stat-block dd { margin: 0; color: var(--parchment); font-size: 0.94rem; } .chip-guide .read-more { display: inline-block; margin-top: 0.4rem; color: var(--bronze-bright); font-family: 'Cinzel', serif; font-weight: 700; text-decoration: none; border-bottom: 1px solid var(--bronze); padding-bottom: 1px; letter-spacing: 0.05em; font-size: 0.92rem; } .chip-guide .read-more:hover { color: var(--gold); border-bottom-color: var(--gold); } .chip-guide .quote { font-family: 'IM Fell English', serif; font-style: italic; font-size: 1.18rem; color: var(--parchment); background: rgba(184, 115, 51, 0.08); border-left: 4px solid var(--bronze); padding: 1rem 1.5rem; margin: 1.5rem 0; } .chip-guide ul { padding-left: 1.4rem; } .chip-guide ul li { margin: 0.3rem 0; } </style> <div class="chip-guide"> <div class="hero"> <div class="eyebrow">A Field Guide · 2026</div> <h1 id="a-field-guide-to-ai-chips">A Field Guide to AI Chips</h1> <div class="subtitle">Stat blocks, lairs and notable specimens for the eight kinds of silicon that power modern AI.</div> <div class="ornament">❦ ❦ ❦</div> </div> <p>Modern AI runs on a small zoo of specialised chips. Each evolved to handle a different workload — training a frontier model, answering a billion queries a day, recognising a face on your phone, keeping a drone alive in the air. This guide catalogues eight of them, with a stat block and a &ldquo;where you&rsquo;ll meet it&rdquo; entry for each. Each section links to a deeper entry for the curious.</p><![CDATA[

A Field Guide · 2026

A Field Guide to AI Chips

Stat blocks, lairs and notable specimens for the eight kinds of silicon that power modern AI.

❦ ❦ ❦

Modern AI runs on a small zoo of specialised chips. Each evolved to handle a different workload — training a frontier model, answering a billion queries a day, recognising a face on your phone, keeping a drone alive in the air. This guide catalogues eight of them, with a stat block and a “where you’ll meet it” entry for each. Each section links to a deeper entry for the curious.

The Roll Call

Chip	Best for	Memory & Interconnect	Cost & Access	Notable Specimens (2026)
GPU	Training + inference	80–192GB HBM3/3e; NVLink 5, PCIe 5	$25–40K each; cloud-only at scale	NVIDIA H100, B200, GB200 NVL72; AMD MI325X
TPU	Hyperscale training	95–192GB HBM; OCS interconnect	Google Cloud only	TPU v5p, v6 Trillium
NPU	On-device AI	Shared LPDDR / unified memory	Bundled in device	Apple Neural Engine (M4), Intel AI Boost (Lunar Lake), Qualcomm Hexagon (8 Elite)
CPU	Orchestration & control plane	DDR5; PCIe 5, CXL	$1–15K; retail	Intel Xeon 6, AMD EPYC 9005
ASIC	Inference at scale; specialised training	Custom HBM / SRAM; proprietary fabric	Cloud-only	AWS Inferentia2, Trainium2; Cerebras WSE-3; Groq LPU; SambaNova SN40L
FPGA	Custom, low-latency, adaptive	DDR/HBM; reprogrammable fabric	$5–50K each; cloud	AMD Versal AI Edge, Intel Agilex 7
Edge AI	Mobile, robotics, IoT	LPDDR; low-power	$50–2000, embedded in product	NVIDIA Jetson Orin, Google Coral, Hailo-8
Emerging	Frontier R&D	Wafer-scale SRAM / photonic / analog	Mostly research, limited cloud	Cerebras (covered above), Lightmatter, Mythic

1 · GPU — Graphics Processing Unit

GPU · The Apex Predator

Class

Parallel beast

Memory

80–192GB HBM3 / HBM3e

Interconnect

NVLink 5, PCIe 5, InfiniBand

Power

350–1200W per die

Habitat

Hyperscale datacenters

Cost

$25–40K per card · cloud rental at scale

Best Prey

LLM training, diffusion, multimodal pretraining

Specimens

NVIDIA H100, B200, GB200 NVL72; AMD MI325X

GPUs are the apex predator of the AI hardware ecosystem in 2026. Originally designed for graphics, they turned out to be ideal for the dense matrix multiplications that dominate neural network training. NVIDIA’s H100 made the LLM era possible; B200 and the rack-scale GB200 NVL72 (72 GPUs treated as one machine, lashed together by NVLink switches) define the current frontier.

The reason GPUs dominate isn’t just parallel processing — it’s the combination of HBM (high-bandwidth memory mounted directly on the chip package), tensor cores (specialised matrix-multiply units), and a mature software ecosystem (CUDA, PyTorch, JAX) that nothing else has matched at scale. AMD’s MI325X is the only serious open-market competitor, and even it ships running CUDA-compatible code through ROCm translation.

The catch: you cannot really buy them. H100s and B200s ship into hyperscaler datacenters first and reach the open market — when they do — through Lambda, CoreWeave, AWS, and friends, rented by the hour at $2–8 each.

→ Full entry: Field Guide · GPUs

2 · TPU — Tensor Processing Unit

TPU · Google's Matmul Colossus

Class

Bespoke matrix engine

Memory

95–192GB HBM (varies by generation)

Interconnect

OCS (Optical Circuit Switching) + ICI

Power

~200–300W per chip

Habitat

Google Cloud (only)

Cost

Cloud rental only

Best Prey

Hyperscale training of Gemini-class models

Specimens

TPU v5p (training), v5e (inference), v6 Trillium

Google designed TPUs in-house to avoid paying NVIDIA’s margins on a workload they knew exactly — TensorFlow matrix multiplications at hyperscale. Each generation has narrowed the gap with GPUs on flexibility while widening it on energy efficiency per FLOP.

The architectural bet is the systolic array: a grid of multiply-add units that pumps data through in lockstep, achieving near-peak utilisation on matmul-heavy workloads. The trade-off is that anything outside that sweet spot (irregular memory access, highly dynamic shapes) runs less efficiently than on a GPU. The OCS-based interconnect lets Google rewire a TPU pod’s topology per job, which matters enormously at the scale of a Gemini training run.

You cannot buy a TPU. They exist exclusively inside Google Cloud, rented by the hour. Gemini was trained on them; many third parties (Anthropic for a stretch, plus enterprise customers) rent slices for their own runs.

→ Full entry: Field Guide · TPUs

3 · NPU — Neural Processing Unit

NPU · The Resident Familiar

Class

On-device specialist

Memory

Shared LPDDR / unified system memory

Interconnect

SoC fabric (on-die)

Power

5–40W

Habitat

Laptops, phones, tablets

Cost

Bundled — no separate purchase

Best Prey

Voice, camera AI, on-device LLMs, Copilot features

Specimens

Apple Neural Engine (M4), Intel AI Boost (Lunar Lake / Arrow Lake), Qualcomm Hexagon (8 Elite, X Elite)

NPUs are the chip type most people interact with every day without knowing it. They live inside the SoC of your phone or laptop, optimised for running already-trained models locally with extreme power efficiency. Voice transcription, Face ID, Pixel’s call screening, the on-device chat in Copilot+ PCs — all NPU workloads.

The defining trait is integer-quantised math (INT8 / INT4) at very low wattage. Where a datacenter GPU might pull 700W to serve a model, an NPU runs a comparable inference on the same model — quantised down — at 5–15W, with the weights sitting in the device’s main memory because there is no discrete accelerator memory to fill.

Microsoft now requires 40+ TOPS of NPU performance for a laptop to qualify as a “Copilot+ PC” — a forcing function that pushed Qualcomm, Intel and AMD into a 12-month arms race. As of 2026, top mobile SoCs ship 50–60 TOPS of NPU performance.

→ Full entry: Field Guide · NPUs

4 · CPU — Central Processing Unit

CPU · The Foundational Workhorse

Class

General-purpose

Memory

DDR5 (system RAM)

Interconnect

PCIe 5, CXL

Power

100–500W

Habitat

Every server, every workstation

Cost

$1–15K, retail and widely available

Best Prey

Orchestration, preprocessing, control plane, small-batch inference

Specimens

Intel Xeon 6, AMD EPYC 9005

CPUs aren’t obsolete — they’re indispensable. Every AI training run needs CPUs to feed data to the accelerators (decompression, tokenisation, augmentation), schedule jobs and run the control plane. The ratio matters: a typical training cluster pairs eight GPUs with one or two CPU sockets.

Modern server CPUs ship with AI-targeted extensions — AVX-512, AMX (Advanced Matrix Extensions), bf16 support — that let them handle small-batch inference and embedding generation reasonably well. For workloads under 7B parameters at low traffic, a CPU is often more economical than a dedicated accelerator.

What CPUs cannot do is train frontier models. The arithmetic density and memory bandwidth needed for LLM pretraining is 10–100× what a CPU delivers per watt. CPUs do the surrounding work; accelerators do the math.

→ Full entry: Field Guide · CPUs

5 · ASIC — Application-Specific Integrated Circuit

ASIC · The Purpose-Bred Specialist

Class

Fixed-function accelerator

Memory

Custom HBM / on-die SRAM

Interconnect

Proprietary fabric

Power

75–450W per chip

Habitat

Hyperscaler clouds (AWS, Cerebras, Groq, SambaNova)

Cost

Cloud rental only

Best Prey

Inference at scale, specialised training

Specimens

AWS Inferentia2, Trainium2; Cerebras WSE-3; Groq LPU; SambaNova SN40L

ASICs are chips designed for one thing and one thing only — and they’re brutally good at that thing. AWS Inferentia2 runs production inference for Anthropic, Amazon search and Alexa at a cost-per-token that beats GPUs. Trainium2 is AWS’s training equivalent, taking aim at NVIDIA’s H100/B200 dominance. Groq’s LPU posts inference latencies — sub-1ms first-token for many models — that GPUs simply cannot match.

The architectural philosophy is “build silicon for the specific math you do most often, throw away the rest.” Cerebras takes this furthest: their Wafer-Scale Engine 3 is a single chip the size of an entire silicon wafer (900,000 cores, 44GB on-die SRAM) that eliminates the multi-GPU communication overhead which plagues distributed training.

The price of specialisation: you cannot pivot. When the dominant architecture changes — and it does (Mamba, MoE, diffusion, JEPA) — ASICs designed for the last era stop being competitive overnight. GPUs hedge their bets; ASICs commit.

→ Full entry: Field Guide · ASICs

6 · FPGA — Field-Programmable Gate Array

FPGA · The Shapeshifter

Class

Reprogrammable logic

Memory

DDR / HBM (model-dependent)

Interconnect

PCIe; custom

Power

50–300W

Habitat

Trading desks, 5G basebands, telecom, occasional inference

Cost

$5–50K each; cloud

Best Prey

Ultra-low-latency inference, custom protocols, evolving workloads

Specimens

AMD/Xilinx Versal AI Edge, Intel Agilex 7

FPGAs occupy a strange ecological niche. Unlike ASICs, their internal wiring is reprogrammable — you can compile a new circuit into them, deploy it, and reprogram it tomorrow. This makes them ideal for workloads that change faster than chip fabrication cycles (years), or where you need an ultra-low-latency that even ASICs struggle to deliver.

In AI specifically, FPGAs are rarely the first choice for mainstream training or inference — they are slower to develop for and harder to program than GPUs. Where they shine: when the model is small enough to fit, the latency budget is brutal (single-digit microseconds), and the workload spec might shift quarterly. Microsoft used FPGAs heavily in early Bing Search ranking and Azure networking; financial firms still run them for inline ML in trading.

For most readers, FPGAs will be a “did you know?” category rather than a chip you’ll ever deploy.

→ Full entry: Field Guide · FPGAs

7 · Edge AI — Mobile, Robotics, IoT

Edge AI · The Frontier Ranger

Class

Embedded inference

Memory

LPDDR, sometimes onboard SRAM

Interconnect

PCIe, MIPI, USB

Power

1–25W

Habitat

Drones, robots, cameras, autonomous systems, sensors

Cost

$50–2000; embedded in product

Best Prey

Real-time inference, computer vision, robotics

Specimens

NVIDIA Jetson Orin, Google Coral / Edge TPU, Hailo-8, Ambarella CV5

Edge AI chips are NPUs’ cousins — same family, different role. Where an NPU lives inside a consumer laptop alongside other compute, an edge AI chip is purpose-built for an embedded device: a security camera, a drone, a forklift, a Tesla.

The defining constraints are size, power and latency. A camera processing 4K video at 30fps cannot afford to ship frames to a cloud GPU; it has to detect motion locally, identify objects locally and signal events within tens of milliseconds — on a few watts, because the device runs on battery or is fanless.

NVIDIA’s Jetson family is the broadest platform — same CUDA software stack as their datacenter GPUs, scaled down to 7–60W. Google’s Edge TPU is the smallest, cheapest and lowest power (Coral USB stick: $40, 2W). Hailo-8 and Ambarella sit in between, targeting industrial and automotive customers.

→ Full entry: Field Guide · Edge AI

8 · Emerging Architectures

Emerging · The Frontier Beasts

Class

Experimental

Memory

Wafer-scale SRAM / photonic / analog

Interconnect

On-wafer / optical / in-memory

Power

Varies wildly

Habitat

Research labs, narrow cloud offerings

Cost

Mostly inaccessible; limited cloud

Best Prey

The next 10× efficiency leap

Specimens

Cerebras WSE-3 (wafer-scale); Lightmatter, Lightelligence (photonic); Mythic, IBM (analog/in-memory)

Three architectures sit at the frontier — promising, but not yet mainstream.

• Wafer-scale (Cerebras): one single chip the size of an entire silicon wafer. Eliminates multi-chip communication entirely; presents the whole system to software as a single device. Already commercial.
• Photonic / optical AI (Lightmatter, Lightelligence): perform matrix math using light interference instead of electricity. Potentially orders of magnitude lower energy per operation; currently limited to inference and constrained models.
• Analog / in-memory compute (Mythic, IBM, several startups): computeinside memory arrays using analog voltage levels. Removes the von-Neumann bottleneck — the constant shuttling of data between memory and compute — entirely. Promising for low-power inference; precision limitations make training hard today.

→ Full entry: Field Guide · Emerging Architectures

Current Industry Reality (2026)

• GPUs dominate training. Every frontier model — GPT-5, Claude, Gemini, Llama, Grok — is still trained on NVIDIA or AMD silicon at hyperscale.
• ASICs are ascendant in inference. AWS reports more than 40% of internal inference now runs on Inferentia and Trainium; Groq leads on latency-critical applications.
• NPUs are exploding on consumer devices. Every premium laptop and phone shipped in 2026 has a 40+ TOPS NPU.
• CPUs remain foundational. No accelerator runs without one.
• TPUs are Google-only. Gemini, Veo and Imagen were all trained on TPU v5p / v6.

Simplified View

Use case	Typical chip
Train a GPT-class model	GPU clusters (or TPU pods if you're Google)
Run ChatGPT-class inference at scale	GPUs + ASICs (Inferentia, Groq, Trainium)
AI on laptop	NPU + integrated GPU
AI on phone	Mobile NPU
Robot or drone AI	Edge AI chips (Jetson, Hailo)
Ultra-low-latency custom AI	FPGA or ASIC

The Industry Trend

The industry is moving from "general-purpose GPU everything" to "specialised chip for each layer of the stack."

Power and inference cost are now the binding constraints. A frontier model serving billions of queries spends more on inference electricity in a year than its entire training run cost. The economics force specialisation: train once on GPUs, serve forever on cheaper inference silicon. Expect the gap between training hardware (still GPU-dominant) and inference hardware (rapidly ASIC- and NPU-fragmented) to widen.

❦ ❦ ❦

]]>Postsartificial-intelligencearchitecturehardwareengineeringKnowledge Basehttps://curiousbit.netlify.app/one-predicts-words-one-maps-meaning/https://curiousbit.netlify.app/one-predicts-words-one-maps-meaning/Tue, 03 Mar 2026 00:00:00 +0000Ajay Walia<style> .two-mech { margin: 2rem 0 2.5rem; border-radius: 14px; overflow: hidden; border: 1px solid #1f3358; background: #0a1424; } .two-mech svg { display: block; width: 100%; height: auto; min-width: 720px; } .two-mech-wrap { overflow-x: auto; } @media (prefers-reduced-motion: reduce) { .two-mech .tm-particle { display: none; } } </style> <div class="two-mech"><div class="two-mech-wrap"> <svg viewBox="0 0 1200 720" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated diagram contrasting the LLM stochastic generation loop with the deterministic embedding similarity pipeline"> <defs> <filter id="tmGlow" x="-50%" y="-50%" width="200%" height="200%"> <feGaussianBlur stdDeviation="3" result="b"/> <feMerge><feMergeNode in="b"/><feMergeNode in="SourceGraphic"/></feMerge> </filter> <marker id="tmArrowBlue" markerWidth="10" markerHeight="10" refX="6" refY="5" orient="auto"> <path d="M0,0 L10,5 L0,10 Z" fill="#60a5fa"/> </marker> <marker id="tmArrowGreen" markerWidth="10" markerHeight="10" refX="6" refY="5" orient="auto"> <path d="M0,0 L10,5 L0,10 Z" fill="#34d399"/> </marker> <marker id="tmArrowAmber" markerWidth="10" markerHeight="10" refX="6" refY="5" orient="auto"> <path d="M0,0 L10,5 L0,10 Z" fill="#f59e0b"/> </marker> <!-- Hidden paths the particles animate along --> <path id="tmPathLLM" d="M 140 240 L 320 240 L 500 240 L 680 240 L 860 240 L 860 320 Q 860 360 820 360 L 540 360 Q 500 360 500 320 L 500 280 L 500 240 L 680 240 L 860 240" fill="none" stroke="none"/> <path id="tmPathEmb" d="M 140 540 L 320 540 L 500 540 L 680 540 L 860 540" fill="none" stroke="none"/> <path id="tmPathEmb2" d="M 140 540 L 320 540 L 500 540 L 680 540 L 860 540" fill="none" stroke="none"/> </defs> <!-- background --> <rect width="1200" height="720" fill="#0a1424"/> <!-- header --> <text x="120" y="60" font-family="'Space Grotesk','Inter',sans-serif" font-size="16" fill="#f59e0b" letter-spacing="3" font-weight="700">TWO MECHANISMS</text> <text x="120" y="100" font-family="'Space Grotesk','Inter',sans-serif" font-size="38" fill="#ffffff" font-weight="700" letter-spacing="-.5">Generation vs Similarity</text> <!-- ════════ LLM PIPELINE (top) ════════ --> <text x="120" y="180" font-family="'Space Grotesk','Inter',sans-serif" font-size="26" fill="#60a5fa" font-weight="700">LLM</text> <text x="120" y="208" font-family="'Inter',sans-serif" font-size="14" fill="#7e95b5" font-style="italic">non-deterministic · sampling</text> <!-- LLM boxes --> <g font-family="'Inter',sans-serif"> <!-- prompt --> <g> <rect x="60" y="200" width="160" height="80" rx="10" fill="#0f1d33" stroke="#3b82f6" stroke-width="2"/> <text x="140" y="234" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">prompt</text> <text x="140" y="258" text-anchor="middle" font-size="13" fill="#7e95b5">"AI is..."</text> </g> <!-- tokenize --> <g> <rect x="240" y="200" width="160" height="80" rx="10" fill="#0f1d33" stroke="#3b82f6" stroke-width="2"/> <text x="320" y="234" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">tokenize</text> <text x="320" y="258" text-anchor="middle" font-size="13" fill="#7e95b5">BPE → IDs</text> </g> <!-- model --> <g> <rect x="420" y="200" width="160" height="80" rx="10" fill="#0f1d33" stroke="#3b82f6" stroke-width="2"/> <text x="500" y="234" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">model</text> <text x="500" y="258" text-anchor="middle" font-size="13" fill="#7e95b5">forward pass</text> </g> <!-- sample --> <g> <rect x="600" y="200" width="160" height="80" rx="10" fill="#0f1d33" stroke="#3b82f6" stroke-width="2"/> <text x="680" y="234" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">sample</text> <text x="680" y="258" text-anchor="middle" font-size="13" fill="#7e95b5">temp / top-p</text> </g> <!-- next token --> <g> <rect x="780" y="200" width="160" height="80" rx="10" fill="#0f1d33" stroke="#3b82f6" stroke-width="2"/> <text x="860" y="234" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">next token</text> <text x="860" y="258" text-anchor="middle" font-size="13" fill="#7e95b5">append, loop</text> </g> </g> <!-- LLM arrows --> <g stroke="#60a5fa" stroke-width="2" fill="none"> <line x1="220" y1="240" x2="232" y2="240" marker-end="url(#tmArrowBlue)"/> <line x1="400" y1="240" x2="412" y2="240" marker-end="url(#tmArrowBlue)"/> <line x1="580" y1="240" x2="592" y2="240" marker-end="url(#tmArrowBlue)"/> <line x1="760" y1="240" x2="772" y2="240" marker-end="url(#tmArrowBlue)"/> </g> <!-- Stochastic loop arrow (next token → model) --> <path d="M 860 280 L 860 340 Q 860 360 840 360 L 520 360 Q 500 360 500 340 L 500 286" fill="none" stroke="#f59e0b" stroke-width="2" stroke-dasharray="5,4" marker-end="url(#tmArrowAmber)"/> <text x="680" y="384" text-anchor="middle" font-family="'Inter',sans-serif" font-size="13" fill="#f59e0b" font-style="italic">stochastic loop</text> <!-- Pulsing rings on each LLM box (highlight as particle passes) --> <rect class="tm-box-pulse" x="60" y="200" width="160" height="80" rx="10" fill="none" stroke="#60a5fa" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="5s" begin="0s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="240" y="200" width="160" height="80" rx="10" fill="none" stroke="#60a5fa" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="5s" begin="0.6s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="420" y="200" width="160" height="80" rx="10" fill="none" stroke="#60a5fa" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="5s" begin="1.2s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="600" y="200" width="160" height="80" rx="10" fill="none" stroke="#60a5fa" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="5s" begin="1.8s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="780" y="200" width="160" height="80" rx="10" fill="none" stroke="#60a5fa" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="5s" begin="2.4s" repeatCount="indefinite"/> </rect> <!-- Animated particle on the LLM path (loops continuously, traverses stochastic loop) --> <circle class="tm-particle" r="9" fill="#60a5fa" filter="url(#tmGlow)"> <animateMotion dur="5s" repeatCount="indefinite" rotate="auto"> <mpath href="#tmPathLLM"/> </animateMotion> </circle> <!-- ════════ EMBEDDING PIPELINE (bottom) ════════ --> <text x="120" y="480" font-family="'Space Grotesk','Inter',sans-serif" font-size="26" fill="#34d399" font-weight="700">EMBEDDING</text> <text x="120" y="508" font-family="'Inter',sans-serif" font-size="14" fill="#7e95b5" font-style="italic">deterministic · geometric</text> <g font-family="'Inter',sans-serif"> <!-- word --> <g> <rect x="60" y="500" width="160" height="80" rx="10" fill="#0f1d33" stroke="#10b981" stroke-width="2"/> <text x="140" y="534" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">word</text> <text x="140" y="558" text-anchor="middle" font-size="13" fill="#7e95b5">"king"</text> </g> <!-- lookup --> <g> <rect x="240" y="500" width="160" height="80" rx="10" fill="#0f1d33" stroke="#10b981" stroke-width="2"/> <text x="320" y="534" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">lookup</text> <text x="320" y="558" text-anchor="middle" font-size="13" fill="#7e95b5">GloVe / SBERT</text> </g> <!-- vector --> <g> <rect x="420" y="500" width="160" height="80" rx="10" fill="#0f1d33" stroke="#10b981" stroke-width="2"/> <text x="500" y="534" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">vector</text> <text x="500" y="558" text-anchor="middle" font-size="13" fill="#7e95b5">fixed dim</text> </g> <!-- cosine --> <g> <rect x="600" y="500" width="160" height="80" rx="10" fill="#0f1d33" stroke="#10b981" stroke-width="2"/> <text x="680" y="534" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">cosine</text> <text x="680" y="558" text-anchor="middle" font-size="13" fill="#7e95b5">vs corpus</text> </g> <!-- similarity --> <g> <rect x="780" y="500" width="160" height="80" rx="10" fill="#0f1d33" stroke="#10b981" stroke-width="2"/> <text x="860" y="534" text-anchor="middle" font-size="17" fill="#ffffff" font-weight="700">similarity</text> <text x="860" y="558" text-anchor="middle" font-size="13" fill="#7e95b5">score · rank</text> </g> </g> <g stroke="#34d399" stroke-width="2" fill="none"> <line x1="220" y1="540" x2="232" y2="540" marker-end="url(#tmArrowGreen)"/> <line x1="400" y1="540" x2="412" y2="540" marker-end="url(#tmArrowGreen)"/> <line x1="580" y1="540" x2="592" y2="540" marker-end="url(#tmArrowGreen)"/> <line x1="760" y1="540" x2="772" y2="540" marker-end="url(#tmArrowGreen)"/> </g> <!-- Pulsing rings on each Embedding box --> <rect class="tm-box-pulse" x="60" y="500" width="160" height="80" rx="10" fill="none" stroke="#34d399" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="4s" begin="0s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="240" y="500" width="160" height="80" rx="10" fill="none" stroke="#34d399" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="4s" begin="0.7s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="420" y="500" width="160" height="80" rx="10" fill="none" stroke="#34d399" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="4s" begin="1.4s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="600" y="500" width="160" height="80" rx="10" fill="none" stroke="#34d399" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="4s" begin="2.1s" repeatCount="indefinite"/> </rect> <rect class="tm-box-pulse" x="780" y="500" width="160" height="80" rx="10" fill="none" stroke="#34d399" stroke-width="3" opacity="0"> <animate attributeName="opacity" values="0;1;0" dur="4s" begin="2.8s" repeatCount="indefinite"/> </rect> <!-- Embedding particle (one-shot left-to-right, restarts cleanly) --> <circle class="tm-particle" r="9" fill="#34d399" filter="url(#tmGlow)"> <animateMotion dur="4s" repeatCount="indefinite" rotate="auto"> <mpath href="#tmPathEmb"/> </animateMotion> </circle> <!-- Annotation under embedding to underscore "one-shot, no loop" --> <text x="500" y="624" text-anchor="middle" font-family="'Inter',sans-serif" font-size="13" fill="#34d399" font-style="italic">one-shot · same input always produces same output</text> <!-- Bottom rule + caption --> <line x1="60" y1="660" x2="940" y2="660" stroke="#1f3358" stroke-width="1"/> <text x="500" y="690" text-anchor="middle" font-family="'Inter',sans-serif" font-size="14" fill="#7e95b5" font-style="italic">Both paths exist in every modern NLP system. Which one you reach for depends on whether the answer needs to be <tspan fill="#60a5fa" font-weight="700">written</tspan> or <tspan fill="#34d399" font-weight="700">found</tspan>.</text> </svg> </div></div> <p>The model is what writes the email. The embedding is what finds the one you wrote last March.</p><![CDATA[

The model is what writes the email. The embedding is what finds the one you wrote last March.

Most modern AI systems are built from two fundamentally different mechanisms, and most confusion about what AI “is” comes from conflating them. LLMs aregenerative: tokens in, tokens out, with the output shaped by the prompt and the sampling settings, varying every time you ask. Embeddings aregeometric: a deterministic mapping from a word or sentence to a fixed vector, where comparisons are positional and identical input always produces identical output. Both are essential. Both are old enough to be uncontroversial. Most useful systems combine them.

What follows is theWeek 6 Graded Mini Project of theIITM Pravartak Professional Certificate Programme in Agentic AI and Applications, used here as a lens for both mechanisms across five hands-on exercises.

The two paths, side by side

The header image above shows the contrast in one frame. The LLM path is a loop with sampling — non-deterministic by design, behaviour controlled by temperature, top-p, and prompt structure. The embedding path is a one-shot lookup followed by a geometric comparison — deterministic, fast, stable.

That single distinction tells you which mechanism to reach for. If the answer needs to be written, generated, synthesized, or improvised, you want the LLM. If the answer needs to be found, ranked, deduplicated, clustered, or routed, you want embeddings. Most production systems use both because most real problems are some combination of “find the right context” and “say something useful about it.”

A quick decision table to anchor the rest of the article:

Problem	Reach for
Semantic search over a corpus	Embeddings
Conversational reply or text drafting	LLM
Near-duplicate detection or content clustering	Embeddings
Summarization of a long document	LLM
Routing a support ticket to the right team	Embeddings + a small classifier head
Question answering grounded in your docs	Both (RAG)
Image or text classification	Embeddings + a categorical head
Translation, rewriting, code generation	LLM

The exercises below show why each row works the way it does.

Exercise 1: Text generation reveals prompt and sampling sensitivity

Section A1 loadeddistilgpt2 through the Hugging Facepipeline API and generated three continuations of the same prompt:

generator=pipeline("text-generation",model="distilgpt2")generator("AI is transforming industries by",max_new_tokens=40,num_return_sequences=3,do_sample=True)

Three continuations came back from the same model, the same prompt, the same call:

“AI is transforming industries by using science to bring people together with a greater understanding of the importance of science. The new book takes an approach to both science and technology, allowing people to focus more more effectively on the basics and to…”

“AI is transforming industries by replacing the manufacturing sector with a manufacturing sector that can be turned into a manufacturing and IT sector by creating new jobs and creating new jobs. The new jobs and investment in the next decade will help spur growth…”

“AI is transforming industries by creating a new, faster, and more attractive way of generating capital and creating jobs for both the United States and Europe. This is an effective new way of doing this.”

Three different stories. None of which the model “knew” — it just produced plausible-sounding next tokens under stochastic sampling. Notice the repetitions (“manufacturing sector with a manufacturing sector”), the loops (“more more effectively”), the empty filler (“a new, faster, and more attractive way of generating capital”). DistilGPT-2 is a small model — these are the artefacts of a system that’s good at local fluency but doesn’t have a strong forward plan.

The headline insight: LLM outputs are statistical, prompt-sensitive, and unrepeatable unless you fix the seed. The same prompt can give you variety (a feature when brainstorming) or drift (a bug when consistency matters).

Exercise 2: Tokenization is where the abstraction begins

This is the section to slow down on. Take the sentence:

“LLMs are powerful tools for natural language understanding.”

A human reads eight words. The model sees ten tokens.

After BPE (Byte-Pair Encoding) with the DistilGPT-2 tokenizer:

['LL', 'Ms', 'Ġare', 'Ġpowerful', 'Ġtools', 'Ġfor', 'Ġnatural',
'Ġlanguage', 'Ġunderstanding', '.']

The stringLLMs doesn’t appear in the model’s vocabulary as a single unit, so it is split intoLL andMs. TheĠ prefix encodes “preceding space” — that’s how BPE preserves word boundaries without a separator character. The period gets its own token.

The mismatch betweenwhat a human reads andwhat the model processes has real consequences:

• Cost is per token, not per word. API billing, latency, and rate limits are all token-denominated. A 1,000-word prompt to a frontier model may bill at 1,300–1,500 tokens depending on language.
• Context windows are token windows. A 4,096-token context holds roughly 3,000 English words. Much less for code (whitespace and symbols inflate counts), much less again for languages with poor vocabulary coverage in the tokenizer.
• Rare strings behave oddly. Brand names, technical acronyms, foreign words, internal jargon — anything outside the trained vocabulary gets fractured. Model behaviour around those fractures is harder to predict, and prompt sensitivity often hides at this layer.
• The same string can tokenize differently with leading whitespace."king" and" king" are different token sequences. That’s why pasted prompts sometimes produce subtly different outputs than typed ones.

Tokenization is the lowest layer of the LLM stack and the one most engineering conversations skip. If you’re tuning prompts and getting unstable behaviour, the first place to look is what your input looks likeafter the tokenizer touches it, not what it looks like in your editor.

Exercise 3: Prompts shape what you get

Section B ran three task-shaped prompts through the same generator, withtemperature=0.8 andtop_p=0.95:

• Summarization — explicit instruction with a 30-word cap.
• Q&A — structured format withQ: andA: markers.
• Creative — open-ended request for a 4-line poem about AI.

The summarization output respected the spirit of the constraint but drifted past 30 words on most runs — DistilGPT-2 is small enough that hard length control isn’t reliable even with explicit instructions. The Q&A output, asked for the capital of Japan, returnedI believe... — the model hedged. A larger model would say Tokyo confidently; a small model produces statistically plausible Q&A-shaped text without strong factual grounding. The creative prompt produced varied and stylistic continuations, but with the lowest grounding: fluency over precision.

Structure compresses the output space the model is sampling from. Vagueness expands it. That single sentence is most of what “prompt engineering” actually is — the rest is technique.

Exercise 4: Word embeddings encode semantic geometry

Pivot to the other mechanism. Section C1 loadedGloVe vectors (glove-wiki-gigaword-50 — 50 dimensions, trained on Wikipedia and Gigaword) via Gensim, then asked for the five nearest neighbours of three words:

Query	Top 5 neighbours (cosine similarity)
king	prince (0.82), queen (0.78), ii (0.77), emperor (0.77), son (0.77)
queen	princess (0.85), lady (0.81), elizabeth (0.79), king (0.78), prince (0.78)
diamond	gold (0.77), diamonds (0.77), gem (0.74), silver (0.72), jewel (0.71)

There is no generation here. Each word is mapped to a fixed 50-dimensional vector, and the “nearest neighbours” are the words whose vectors sit closest in that space by cosine similarity. The geometry was learned by training on co-occurrence — words that appear in similar contexts end up in similar positions. That’s whyking andprince are nearest neighbours, whyqueen pulls inelizabeth (the corpus has plenty of references to Queen Elizabeth), and whydiamond cleanly resolves to a jewellery cluster.

The classicking − man + woman ≈ queen analogy works in this same space; the lab didn’t run it, but the geometry is there. Embeddings don’twrite anything — theyplace things near other things. That single property is what makes them the backbone of semantic search, retrieval, deduplication, and recommendation.

Exercise 5: Sentence similarity from averaged word vectors

Section C2 extended the geometry to sentences. Five short sentences across two topics — AI/ML and jewellery — were averaged into sentence vectors (mean of their word vectors, with simple lowercase tokenization), then compared with cosine similarity.

Plotted in 2D via multidimensional scaling on the cosine distances, the clustering is unambiguous:

The numerical version:

	AI/support	ML/fraud	Jewellery	Neural/medical	Luxury/rings
AI/support	1.00	0.84	0.60	0.80	0.50
ML/fraud	0.84	1.00	0.73	0.83	0.62
Jewellery	0.60	0.73	1.00	0.58	0.88
Neural/medical	0.80	0.83	0.58	1.00	0.56
Luxury/rings	0.50	0.62	0.88	0.56	1.00

Within-cluster pairs sit at 0.84–0.88. Cross-domain pairs sit at 0.50–0.62. The grouping is exactly what you’d want a retrieval system to do.

Three caveats worth naming, because they explain why modern retrieval doesn’t actually use GloVe averages:

• Averaging discards word order. “Dog bites man” and “man bites dog” produce identical sentence vectors. For most retrieval that’s tolerable; for anything where syntax carries the meaning, it isn’t.
• Transformer encoders fixed this. Models like BERT, RoBERTa, and their descendants producecontextual embeddings — each token’s vector depends on the tokens around it. Pool those across a sentence and you get a representation that respects word order and disambiguates polysemy.
• Sentence-BERT and friends made it production-grade. SBERT (and successors like OpenAI’stext-embedding-3, Cohere’s embeddings, Voyage, etc.) trained encoders specifically for sentence-level similarity. That’s the difference between “the demo works on five sentences” and “you can index a million documents and search them in milliseconds.”

GloVe averaging is a baseline. It’s the right baseline to start with, because it lets you see the geometry without the architecture getting in the way. Production systems start from this picture and replace the lookup step.

When both mechanisms meet

The final exercise sits at the intersection.distilbert-base-uncased-finetuned-sst-2-english is a transformer encoder (an embedding model under the hood) with a classification head fine-tuned for sentiment. Run it on three workplace-themed inputs:

Input	Label	Score
“The chatbot reduced ticket resolution time by 40% this quarter.”	POSITIVE	0.9962
“Our deployment failed repeatedly and customers were upset.”	NEGATIVE	0.9997
“The new recommendation engine is acceptable but needs tuning.”	NEGATIVE	0.9898

The third row is the interesting one, and it’s worth unpacking because it points at a problem that turns up in every enterprise deployment of pretrained models.

“Acceptable but needs tuning” is, in workplace context, alukewarm-positive — closer to “approved with caveats” than “this is bad.” The classifier scored it NEGATIVE with 0.9898 confidence. Three things are happening at once:

• Domain mismatch. The model was fine-tuned on SST-2, which is movie reviews. “Needs tuning” reads negative there. In an engineering team’s language, “needs tuning” is constructive — the same words have different sentiment loadings in different domains.
• No calibration on workplace text. The score is 0.9898 — extreme confidence — for what should be a borderline case. Pretrained classifiers tend to be miscalibrated on out-of-distribution inputs: they’re not just wrong, they’re confidently wrong. Calibration techniques (temperature scaling, Platt scaling, conformal prediction) exist for exactly this.
• Weak supervision is the practical fix. When you can’t fine-tune (no labelled data, no budget, no time), the durable answer is to treat the classifier as one signal among several — combine it with rules, keyword filters, or a second model — rather than trusting any single number above the threshold.

Architecturally, the lesson generalises across all three Section D variants. Generation is “embedding + decoder loop.” Classification is “embedding + categorical head.” Retrieval is “embedding + cosine.” Same underlying mathematical object, different output shapes. The architectural choices around the embedding determine what the system does — and where it fails when you take it out of the domain it was trained on.

Closing observations

Three things that generalise beyond this lab.

Tokenization is where most LLM cost and quirks actually originate. It’s the lowest layer of the stack and the one most engineering conversations skip. If you’re tuning prompts and getting unstable behaviour, the first place to look is what your input looks like after the tokenizer touches it.

Embedding-based similarity is older, cheaper, and more deterministic than people remember. Before reaching for an LLM call to compare two pieces of text, embed them and compute cosine. It’s milliseconds, free, and stable. A surprising fraction of “AI features” are really embedding lookups with a confidence threshold.

Generation and similarity sit next to each other. They are not competitors. RAG is the obvious example — embeddings retrieve, the LLM generates the answer grounded in what was retrieved. TheWeek 15 RAG chatbot post is what these two mechanisms look like wired together for production.

One predicts words. One maps meaning. Knowing which one to reach for is most of the job.

]]>Postsartificial-intelligencellmmachine-learningengineeringKnowledge Base