Helena Tan | Enterprise AI @_HelenaT

Strongly agree. The PMs who get the most out of coding agents write the sharpest specs; the agent is only as good as the instruction.

Zara Zhang @zarazhangrui

2 days ago

"The market value of writing has gone way up" (per a PM at Anthropic) Good writing/clear articulation is required for both building a product (being able to steer models effectively) and building an audience Writing well has historically been undervalued in tech. Time to change

9 22 222 20K 157

I started noticing this recently. I sometimes got worried about adding too much work to the agent before the subagents finish the previous one or whether that will confuse the subagents.. would be great to see a tool that helps providing easier visibility on the status of each subagent!

@clairevo I completely agree with you. Investing in AI without changing the organizational structure or process will not lead to much ROI. Out of curiosity, what are the best practices (org structure /process) you are seeing?

I spent the weekend with dozens of VP+ leaders on how to ai pill their team, and here were the things that hit them hardest: - yes, you are behind - most are willing to add tools and spend tokens. few are willing to kill process & change jobs - a leader’s lack of ai hard skills is a drag on their org - teams that swallow the pill of investing in internal devx get ahead faster - you think you can’t afford to dedicate time to AI, but teams are getting massive gains w just 1-2 people - you’re underspending tokens - if you think tokens are expensive, tally up the cost of your “alignment” meetings - design is getting left behind - your engineers don’t know these new tools as well as you think - competition is not about product features but about how whole teams operate - you were unprepared for the last 2 years. Don’t be unprepared for the next

@contextconor @hyperspell @maebert Proud of you! 👏👏👏upwards and onwards! 🚀🚀🚀

@mvanhorn More memory reads like more context but it's mostly noise the model re-ranks every turn. Well-curated skills is a good way to address the problem.

@levie One source of token costs come from retries, when an agent re-fetches context it already had permission to cache. How well an agent can optimize re-deriving the same answer can be a differentiator.

@aakashgupta I agree that the bottleneck has moved, in addition to org chart, the bottleneck is also on evals. When anyone can ship in an afternoon, we need easy-to-make and robust evals to determine whether the output is actually right, this is still the hard part.

@jxnlco Will the model propose a stopping condition and let me approve it? Otherwise every /loop feels like runaway token bill!

Models are rented, but the learning loop should be owned. The harder question is how to build one for the enterprise. Here are the areas that we need to get very good at. - Get good at private evals. How can we turn a veteran's tacit judgment into outcomes a machine can grade? This is what makes "better against business outcomes" real. - Make institutional memory queryable, but treat it as an enterprise permission problem. The moment the loop encodes one team's knowledge, we need to figure out who can access it. Enterprise needs to protect it from being a data-leak surface. - Keep the loop model-agnostic. We need to keep the evals, golden sets, memory, and tools living outside any one model, so we can swap a generalist and keep the company veteran.

Satya Nadella @satyanadella

2 weeks ago

x.com/i/article/2065…

3K 8K 41K 66.4M 57K

@mvanhorn @colinraney The enterprise version of this is even stronger: a PR'd skill is auditable and permission-scoped, while the accumulated memory can be a data-residency liability.

@jxnlco How do you scope what a computer-use agent can touch? So that it can do the magics while still within the appropriate guardrails. In the enterprise land, an agent driving a browser can hit anything the session can, which creates a lot of hesitation on adoption.

This speaks to the deep needs/obstacles from Enterprise adoption of AI. In enterprise, the "services" part needs to solve the permission model, data residency, and integration glue no horizontal tool ships by default. Customers pay for outcomes because the last mile, who can touch what data where, is complex, unique to each enterprise.

This speaks to the deep needs/obstacles from Enterprise adoption of AI. In enterprise, the "services" part needs to solve the permission model, data residency, and integration glue no horizontal tool ships by default. Customers pay for outcomes because the last mile, who can touch what data where, is complex, unique to each enterprise.

@arizeai @dat_attacked wondering how you'd handle transcript grading in addition to LLM-as-judge that just grades the final output.

@levie The long-running part is where enterprise demands the next level of harness on Agent Identity. Heavy tool use means each agent needs its own scoped identity, not a shared service account. Data residency also gates the deployment, as agent gets more capable on long-running tasks.

@petergyang Or rather, we need to put a higher emphasis in grading the trace during the eval process. So we don't just grade the answer, but how the LLM gets to the answer. In this case, the drastic reverse after a small push back should show up in the eval and being evaluated accordingly.

@OfficialLoganK On-device is also a data residency story. For some use cases that cannot send data to frontier model APIs, a capable local model isn't a downgrade, it's the only deployable option!

It is super exciting to see Claude's answer to one of the common enterprise questions: does the agent act as the user, a service account, or as itself? Good to see access treated as a first-class decision and the Agent Identity is flexible and intuitive enough to handle various scenarios on slack, and with 3rd party partners like Box.

Really cool walk-through! Like Jiaona said in the video, top 1-2% AI-pilled employees build cool things with AI, but bringing the impact to the rest of 98-99% of the company is often hindered by 1) skills tend to be domain-specific, how to make sure everyone finds and uses the best skills, and not get confused by other skills that do similar things 2) how to make sure right people have the right access to skills etc