AI Agent 的下一道门槛：让网站允许它们进入

Meta 的 Muse、Instinct、ChatGPT 的 Dots 等个人 AI Agent，正掀起新一波面向消费者的 AI 浪潮：它们不再只是回答问题。这些 AI Agent 可以代表用户实际完成任务，例如预订航班、预约晚餐、购买杂货等，而且用户无需具备足够的技术能力，在自己的电脑上部署 OpenClaw 或其他 Agent。

但采用这些 Agent 的消费者经常会遇到一个问题：请求另一端的网站会阻止 AI 完成任务。

最近，Amazon 开始禁止 Meta 的 Muse AI Agent访问其零售网站，这意味着该 Agent 无法再浏览 Amazon 的商品目录或在其中购物。

不过，这一问题并不限于 Amazon 或 Muse。社交媒体上的用户投诉显示，他们的 Agent 经常遭遇类似封锁。其中部分是像 Amazon 那样有意为之，另一些则源于网站用于防范垃圾信息和恶意活动的传统反机器人措施。

终端用户并不清楚，这种封锁究竟是网站所有者无意造成，还是刻意采取的措施；这使他们同时对服务提供商或零售商，以及试图促成交易的 Agent 感到沮丧。

例如，TechCrunch 在社交媒体上看到少量的投诉，称 Muse 无法在 Walmart 的零售网站完成购买。（NBC 也报道，Walmart 在测试这款 AI Agent 时曾遇到这一问题。）

但 Walmart 发言人告诉我们，这些情况并非有意造成。事实上，Walmart 是 Muse 的合作伙伴，双方合作关系已于 9 月在 Meta Connect 开发者大会上公布。该公司表示，希望出现在消费者所在的地方，而这如今也包括通过 AI Agent 提供服务。

Walmart 用户似乎经常遇到的问题（见下文）是，网站会显示一个按钮，要求访问者点击以验证自己是人类。如果这一流程被中断，验证可能失败，Agent 就会被系统踢出。

这表明，我们目前用于验证人类身份、保护网站免受垃圾信息和机器人侵扰的技术，或许并不适合正在迈入的 Agent 优先网络时代。

为了解决这一问题，包括 Meta、Walmart、Stripe、Sierra、Genesys、Rocket、NiCE 和 Decagon 在内的行业伙伴，已经开始制定一项开放标准。该标准将规定 AI Agent 如何与企业通信，帮助区分代表用户行动的“好机器人”和恶意机器人。Meta 在一篇博客文章中表示，这一协议将专门聚焦于在线商业场景中的 Agent 对 Agent 通信。

与此同时，消费者的困惑仍在不断增加。

尽管 Walmart 至少正尝试通过 AI Agent 提供可访问性，但其他公司对这一想法并不热衷，或尚未完全接受这项技术。

例如，航班预订目前是 Agent 被重点推广的主要应用场景之一，但许多人报告称，航空公司拒绝了其 Agent 发出的请求。

例如，Delta 告诉我们，该公司正在采取措施保护客户免受未经授权的自动化活动影响；如果未来向 AI Agent 开放，也必须“将安全和客户体验纳入考量”。

Delta 全球传播总经理 Heena Chavda 在回应我们关于该公司是否如部分用户所称正在封锁 Agent 的提问时表示：“目前，Delta 尚未建立合作关系或集成方案，以支持第三方 AI Agent 代表客户在我们的数字平台上搜索或预订 Delta 航班；但我们会继续评估，包括外部 AI Agent 在内的新技术，如何提升客户与 Delta 互动的方式。”

United 的表态不如 Delta 直接，但一名发言人向我们指出其《使用条款》中的相关内容：用户同意“未经 United 事先书面许可，不得使用任何机器人、网络爬虫、其他自动设备或人工流程监控或复制本网站或其中包含的内容，也不得将其用于任何其他未经授权的目的”。（United 没有回应关于其是否封锁了 Muse 等特定个人 AI Agent 的后续问题。）

早期采用者在社交媒体上的投诉提到，据称有多个品牌正在封锁 AI Agent，包括 Yelp、eBay、Zillow、Pizza Hut、Adidas，以及多家航空公司。一些人认为，这类封锁活动近期似乎有所增加。

还有两名用户称，eBay 因他们使用 Agentic AI 而暂停了他们的账户。另一名用户则抱怨，Google 在 Instinct 帮其清理 Gmail 收件箱时，将 Instinct 踢出了系统。

尽管业界正在推动 Agent 为用户预订餐厅座位和处理其他繁琐任务，Yelp 告诉我们，除非 Agent 通过其数据授权计划付费访问 Yelp 的内容和数据，否则该平台不允许非人类流量。因此，试图通过 Yelp 网站获取报价、将用户加入候位名单或代为预订餐桌的 Agent，如果没有与 Yelp 建立正式合作关系，很可能会失败。

eBay 告诉 TechCrunch，其做法并非禁止所有第三方购物 Agent 完成购买。相反，其政策限制的是未经授权的 Agent 及相关行为，例如自动化抓取和模型训练。

截至发稿时，Adidas、Zillow 和 Pizza Hut 要么未作回应，要么拒绝置评。

这一领域的一家重要公司是 Cloudflare。该公司为网站提供保护，以应对可能利用其数据进行训练的 AI Agent 及其他自动化活动。一些人怀疑Cloudflare及其他类似的内容分发网络（CDN）目前正在干扰 Muse 流量。这被归因于该公司于 9 月 15 日对爬虫默认设置作出的调整：该调整允许网站所有者屏蔽 AI 训练活动，同时允许其他类型的机器人访问。原本因安全原因设置为屏蔽 AI 机器人的现有网站，随后被调整为在所有含广告的页面上屏蔽 AI Agent。

不过，当我们询问 Cloudflare 是否掌握有关 Muse 等个人 AI Agent 被封锁情况的信息时，该公司表示，目前没有可分享的具体数据。该公司转而建议我们查看其免费的公共数据中心 Cloudflare Radar。该网站显示机器人活动的增长情况——Cloudflare 此前也曾对此发表评论——但无法帮助区分好机器人和坏机器人。

Cloudflare 对缓解网站上的 AI 流量也有既得利益，因为它目前运营着一个市场，要求 AI 机器人为其访问的数据付费。该公司最近还开始测试一款专为 AI Agent 打造的新网页浏览器。

鉴于这一领域正在快速演变，Meta 为 Muse 访问权限建立特定品牌合作关系的决定似乎是正确方向，其聚焦新标准的决定也是如此。通过这种方式，Meta 可以向客户保证，至少其 Agent 在这些合作伙伴的网站上是受欢迎的——因此，任何访问问题都可被作为可报告的 Bug 处理。

Meta 还在 Muse 应用内维护了一份“连接器”（即合作伙伴）列表，并且该列表持续增长。Meta 新近宣布的许多合作伙伴尚未出现在这份列表中。但考虑到 Muse 的快速扩张，其服务现已包括面向小企业的合作工具，Muse 的连接器列表可能很快会进行一次更大规模的重组。

Meta 在宣布制定新标准的计划时表示：“拒绝个人 Agent，就意味着拒绝其背后的客户。我们更愿意与企业合作解决根本问题，而不是眼看它们失去这名客户。存在一条对用户和企业都有利的路径：它始于清晰的规范，为企业提供更可预测的控制能力，并逐步发展为让 Agent 更高效协作的方式。”

Personal AI agents, like Meta’s Muse, Instinct, ChatGPT’s Dots, and others, are kicking off a new wave of consumer AI that involves more than just responding to queries. These AI agents can actually get things done on users’ behalf, like booking flights, making dinner reservations, ordering groceries, and more, all without having to be technical enough to set up OpenClaw or some other agent on your own computer.

But consumers adopting these agents are often running into issues when the website on the other end of their request blocks the AI from completing the job.

Most recently, Amazon began blocking Meta’s Muse AI agent from its retail site, meaning that the agent could no longer browse or make purchases from its product catalog.

However, the problem is not limited to Amazon or Muse. User complaints across social media indicate their agents are often running into similar blocks. Some of these are intentional, like Amazon’s, while others are the result of websites’ traditional anti-bot measures designed to protect against spam and malicious activity.

What’s not clear to the end user is whether or not the block is accidental on the website owner’s part, which leaves them frustrated with both the service provider or retailer and the agent trying to facilitate the transaction.

For instance, TechCrunch has seen a handful of complaints on social media claiming that Muse couldn’t complete a purchase on Walmart’s retail website. (NBC also reported that this was a problem Walmart had faced in its tests of the AI agent.)

But a spokesperson for Walmart told us these weren’t intentional. In fact, Walmart is partnered with Muse, as was announced at Meta’s Connect developer conference in September. The company said it wants to be where its customers are, and that now includes being available through AI agent experiences.

The issue that Walmart customers often appear to be running into (see below) is that the website presents a button that has to be clicked to verify the visitor is human. If that experience is interrupted, the check could fail, and the agent gets booted out.

It’s an indication that the current technology we use to verify humans and keep websites safe from spam and bots may not be the best fit for the agent-first era of the web we’re now moving into.

To address this problem, industry partners including Meta, Walmart, Stripe, Sierra, Genesys, Rocket, NiCE, and Decagon have begun working on an open standard that would dictate how AI agents can communicate with businesses, helping to separate the good bots acting on behalf of the user and the bad. This protocol will be specifically focused on agent-to-agent communication for the purposes of online commerce, Meta said in a blog post.

In the meantime, consumer confusion abounds.

Although Walmart is at least trying to be accessible via AI agents, other companies aren’t so keen on the idea or haven’t fully embraced the technology.

For example, flight booking is currently one of the main use cases being positioned for agents, but many people have reported issues with airlines rejecting their agents’ requests.

Delta, for instance, told us that the company is taking steps to protect customers from unauthorized automated activity, and that if it were to open up to AI agents, it would have to be done with “security and the customer experience in mind.”

“While Delta does not currently have a partnership or integration that enables a third-party AI agent to shop for or book Delta flights on a customer’s behalf on our digital platforms, we’re continuing to evaluate how new technology, including external AI agents, could enhance the way customers engage with Delta,” said Delta’s general manager of Global Communications, Heena Chavda, in response to our questions about whether or not it was blocking agents, as some users complained.

United was not quite as direct as Delta, but a spokesperson pointed us to the part of its Terms of Use policy, which says that the user agrees “not to use any robot, spider, other automatic device, or manual process to monitor or copy this website or the content contained therein or for any other unauthorized purpose without United’s prior written permission.” (United didn’t respond to a follow-up question about whether it was blocking specific personal AI agents, like Muse.)

Early adopters’ complaints across social media have referenced a number of brands that are reportedly blocking AI agents, including Yelp, eBay, Zillow, Pizza Hut, Adidas, and various airlines. Some believe that the blocking activity seems to have recently increased.

A couple of people even said eBay suspended their accounts because of their use of agentic AI. Another complained that Google kicked out Instinct while it was cleaning up their Gmail inbox for them.

Despite the push to have agents book tables at restaurants and other tedious tasks, Yelp told us that it doesn’t permit non-human traffic on its platform unless the agent has paid to access Yelp’s content and data through its data licensing program. That means agents trying to use its site to get a quote, add a user to a waitlist, or book them a table will likely fail if they don’t have a formal partnership with Yelp.

eBay told TechCrunch that its approach is not to prohibit all third-party shopping agents from completing purchases. Instead, its policies restrict unauthorized agents and actions, like automated scraping and model training.

Adidas, Zillow, and Pizza Hut either didn’t respond or declined to provide a comment by the time of publication.

One major company involved in this space is Cloudflare, which offers sites protection from AI agents that could be training on its data and managing other automated activity. Some suspect that Cloudflare and other content delivery networks (CDNs) like it are currently disrupting Muse traffic. This is attributed to a change to crawler defaults on September 15, which allows site owners to block AI training while allowing other types of bots. Existing sites that were previously set to block AI bots for security reasons were then shifted to block AI agents on any pages that had ads.

However, when we asked if Cloudflare had any insight into what was happening with the blocking of these personal AI agents, like Muse and others, the company said it didn’t have any specific data to share at this time. It pointed us instead to its free public data hub, Cloudflare Radar. The site shows the increase in bot activity (which Cloudflare has also commented on previously), but it doesn’t help differentiate between the good bots and the bad ones.

Cloudflare also has a vested interest in mitigating AI traffic to websites, as it now runs a marketplace where AI bots have to pay for the data they access. It recently began testing a new web browser built specifically for AI agents.

Given the rapidly evolving space, Meta’s decision to form specific brand partnerships for Muse access seems to be the right way to go, as is its decision to focus on new standards. In doing so, it assures its customers that its agent, at least, is welcome at these partners’ sites — so any issues with access would be reportable bugs.

Meta also maintains a list of “connectors” (a.k.a. partners) inside the Muse app, which continues to grow. Many of Meta’s newly announced partners haven’t yet shown up in this list. But given Muse’s rapid expansion, which now includes partnered tools for small businesses, a bigger reorganization of Muse’s connector list is likely soon to be underway.

“Turning away a personal agent means turning away the customer behind it. We’d rather work with businesses to address the underlying concerns than see them lose that customer,” Meta said in its announcement about the newly announced plans to develop a standard. “There is a path that is good for users and good for businesses. It starts with clear norms, gives businesses more predictable control, and evolves toward a more efficient way for agents to work together.”