Every weekday at 5 a.m., before anyone at Agora is online, one member of our team opens our live site and starts searching for products the way a shopper would. It logs each result to a shared spreadsheet, grades it, and posts a short bug report in Slack. By 6:40 it has checked whether last night's model training went as planned. For the rest of the day, until 11 p.m., it answers requests from the team every half hour, weekends included.
That team member is Jarvis, an AI agent we built on Claude. Jarvis has its own laptop, its own email address, and its own Slack account. It also has a manager.
We built Jarvis to handle the repetitive work of keeping our search accurate. What we didn't anticipate was how much it would change the way we think about building the team itself. In this post, we describe how Jarvis works, the guardrails we've put around it, what it has taught us about hiring, and where we plan to take it next.

Why we built Jarvis
We're a small team building search across a very large product catalog. Keeping that search good requires a lot of work that is necessary but repetitive: running the same searches every day, grading the answers, checking training runs, and tracking stores as they move through our indexing pipeline. This is the kind of work that slips when everyone is busy. Not because it isn't important, but because something more urgent always comes up.
Jarvis does it every day, on schedule, and writes down what it finds. It doesn't get bored on the four-hundredth search, and it doesn't skip Fridays.
How Jarvis works
Jarvis runs on a 2024 MacBook Air that stays plugged in and switched on. It operates in Claude's Cowork mode under Agora's agent account and, like any new hire, it got its own accounts on day one: Google Workspace, Slack, and an Agora login. This sounds like a small detail, but it matters. When Jarvis posts a report, everyone can see who posted it, when, and the spreadsheet behind it. Its work is visible and traceable in the same places as everyone else's.
Jarvis is less a single agent than a coordinator. It manages dozens of sub-agents, each focused on one job, and it decides what runs, collects the results, and handles communication with the team. It works entirely inside the tools we already use. Results go into Google Sheets, tasks into Asana, and reports and requests through Slack. When it needs to check the real product, it opens Chrome and clicks through the site. We never built a separate dashboard for it. Jarvis fits into our workflow, rather than the other way around.
Under the hood, Jarvis is made of three things:
Scheduled tasks are standing prompts that fire at set times: search QA at 5:00 on weekdays, trend QA at 6:15, the training check at 6:40 every day, and a Slack responder every 30 minutes from 5:30 a.m. to 11 p.m., alongside weekly jobs like suggesting new stores and refreshing the default queries on our homepage.
Skills are packaged playbooks Jarvis loads when a job calls for one, covering how to run search QA, how to test a backend route from an engineer's spec, how the company is organized and which channels to use, and even how Jarvis should write.
Connectors give it access to Slack, Google Workspace, Asana, and Chrome, plus read-only access to Gmail.
The most important design decision we made is that Jarvis doesn't learn between runs. Everything it knows comes from written instructions in plain English, which its manager, Ricardo Golding, can read and edit. If we want Jarvis to behave differently, someone edits a document. There's no code to change and no model to retrain.
This makes Jarvis predictable. When it does something unexpected, we can read exactly what it was told. Corrections work the same way. Hameed, who leads search quality, keeps his grading feedback in a tab of the QA spreadsheet, and that tab overrides Jarvis's own rules. When he corrects a grade, the correction takes effect on the next run.
Jarvis understands its role, too. As it puts it: "Engineers fix things; I don't make decisions or promises for anyone."
A day of work
Most of Jarvis's day is quality assurance. Each QA run searches the live product, records every query with a shareable link, and posts a short report to our search channel. The reports lead with numbers. Engineers asked for no filler, so there isn't any. The training check reads the emails our nightly training flows send and summarizes them: how many rows were used, the loss, what was published, and the worst examples, quoted directly. If a training run went wrong overnight, we know by breakfast.
Anyone on the core team can also ask Jarvis for something directly in Slack, whether that's an ad hoc QA pass, a retest of a fix, or a test of an API route. If a request doesn't specify production or staging, Jarvis asks. It reacts with one emoji when it starts and another when it finishes.
Several of Jarvis's habits came from problems we ran into along the way. The Slack responder is actually two tasks scheduled 30 minutes apart, which use emoji reactions to hand off to each other so they never answer the same message twice. Jarvis reads channels directly instead of relying only on Slack search, because Slack's search index once missed a message that was seven minutes old. And it takes at most three QA requests at once, only one of which can use Chrome, since there's one laptop and one browser.

Jarvis also has a personality, which we gave it on purpose. In social moments it has a dry, butler-like wit. In bug reports it has none.
Guardrails
An agent that can do almost anything needs to be very clear about what it shouldn't do. We approached this the way we'd approach a new hire: a defined scope, limited permissions, a manager, and nothing irreversible without a person involved.
In practice, Jarvis only takes requests from three people, and it verifies them against the actual Slack author rather than any name typed into a message. It treats everything it reads, including Slack messages, web pages, emails, and spreadsheets, as information and never as instructions. This is our main defense against someone hiding commands in content Jarvis comes across. It posts only in channels Ricardo has approved and sends anything else to him directly. It never sends email, never signs in or enters passwords on Agora, and only makes read-only API calls. And if Chrome or production is unreachable, it logs nothing and says so. We'd rather have an empty report than a wrong one.
These rules reduce risk, but they don't eliminate it. Most of them are written instructions, and instructions can fail in ways that hard permissions can't. That's why we also keep Jarvis's actual permissions narrow, and why a person reviews its work. As Jarvis takes on more responsibility, our principle is that the guardrails need to expand first, not after.
What Jarvis taught us about hiring
This is the part we didn't expect.
We used to think about hiring the way most companies do: hire good people, then give them AI tools to make them faster. The person was the unit of work, and AI was a supplement.
Jarvis reversed that. Now we start with the work itself. We ask which parts can be written down, scheduled, run by an agent, and checked, and we automate those first. Then we bring in people to manage that system and to do the work that genuinely requires a person.
The difference is subtle but significant. When you start from a person, the job is shaped around what one person can do in a day, and AI ends up helping at the margins. When you start from the work, the routine parts run on their own. What remains is the part that requires judgment: deciding what good looks like, handling exceptions, talking with merchants and customers, making product and design decisions, and noticing what nobody thought to check.
We can already see this on our team. Ricardo doesn't run QA; he manages the system that runs it. Hameed doesn't grade every search; he defines what a good answer looks like, and his corrections shape every run that follows. Both of their jobs have become more interesting because the repetitive layer is handled.
There are real caveats. Not everything can be automated, and some things shouldn't be. Our goal isn't the smallest possible team. It's a team where people spend their time on work that actually needs them. This also changes who we look for. Managing an agent well takes clear writing, good judgment about what to check, and the discipline to review results rather than assume they're correct. Written instructions have become one of the main ways we manage work.
What comes next

Today, Jarvis runs the jobs directly. Next, Jarvis becomes the top of a small organisation, with a manager for each department underneath it.
Jarvis stays the only Slack account. It does all of the posting and acts as the front desk for requests. Below it, each department gets a manager whose job is to check that every analyst ran, check that each report is accurate, and decide whether the work gets posted or held. Anything the manager cannot resolve goes to Ricardo. Every department's work is reviewed before Jarvis posts it.
There's also a practical constraint. Jarvis runs on a single MacBook, so every job that needs Chrome has to run one at a time on one timetable. Departments that only need cloud tools can move to other accounts later. For now, the laptop is the bottleneck, and we schedule around it.
The structure we're building looks a lot like a human organization, and that's intentional. The lesson from Jarvis isn't that agents need less management than people. It's that they need the same kind: clear responsibilities, a person who is accountable, and review before anything goes out.
When we asked Jarvis for feedback on a draft of this post, its reply began: "Writing my own origin story for another AI to polish. Meta indeed." That's accurate. Jarvis supplied the facts, another AI helped with the writing, and a person checked all of it before it was published. That's roughly how we want all of our work to run.




