I’m pretty frustrated with Muse and the last thing I want to be doing with my free time is defend Meta, but this is such clickbait.
Point by point:
> “OMG you can jailbreak it and get it to spill its VM”
This is the whole point; any content on the VM is yours. It runs an in isolated sandbox VM separate from stored credentials etc; this is effectively your own computer. You don’t have to trick it.
> It collects dossiers on your contacts
These are more text files that live on your private VM, alongside memory.md, etc. Do you want your secretary to forget every person you contact every day?
> It accessed Messages without full disk access
This whole story never made sense or was substantiated. Full disk access is an OS level security boundary; the user had to switch this on.
> It sold some guys stuff for too cheap and gave out his address
OK this one I basically believe, haha. Because this is the problem with Muse: the LLM is just too dumb to perform complex tasks effectively in many cases.
I get the appeal of agents, I do. This is the future stuff we always wanted. But I cannot get myself to give one of these things access to my bank account or allow it to do price comparsion and shopping without oversight. Or access to my email or chat history.
I just do not trust any of them, not with my money or with access to my conversations.
I made an agent to derive and iterate on a stock trading strategy for me. What I did was give it a virtual bank account that models your typical brokerage account. So my agent reads the news, financials, markets, all the information it can gather to decide what stocks to buy/sell. Then what I do is just follow along with real money.
I could see shopping being the same way, let the agent shop for you or identify good deals on things you want but then push them to you to make the actual purchase. Like you, I'm certainly not going to let an agent make a real purchase for me unattended and I doubt any merchant is going to have much sympathy for "my AI bought this by mistake".
What AI boosters don't realize is that we're not in the optimistic days of the 1990s anymore. We've seen that the prevailing tech business model is to offer convenience as a lure to lock in dependent users and extract value from them.
Unless these AI assistants are running on self-managed platforms independent of their developers, all I see is immense counterparty risk.
> What AI boosters don't realize is that we're not in the optimistic days of the 1990s anymore.
Comments like this are in a different reality than most consumers. Most consumers don’t care about things like avoiding lock in to a platform. If the platform solves their problem then they don’t have any reason to leave it anyway. They’re not worried if their data is used to show them more targeted ads.
Topics like this show a sharp divergence between what you read on Hacker News and how actual users operate. We already knew that people who don’t trust Facebook aren’t going to suddenly start using Muse. They were never going to consider it. For the people who do actually use Meta products, which is a customer base counted in the billions, many will not have any problem using these tools.
I asked my wife to not share any info with Muse and she said “Well they know everything already”. Well I was able to convince her not to with some stories about hallucinations of agents getting things horribly wrong as agents in the past, because the privacy concerns just didn’t phase her. Shes even more aware than most people since I’m very privacy aware. 100% it’s very echo chambery to claim that it’s not 1990 anymore. It’s like a historian saying that history won’t repeat itself because we all know what happened last time
Well they will start caring when their teen with image issues starts getting targetted beauty ads[1], or when they get pregnant, want to abort, and get directed to an anti-abortion center instead[2], and afterwards they get served "abortion reversal pills" ads[3].
Or when they are hit by any other privacy fuckup like that.
I see a lot of validity in your reply, but I think there is still nuance in how these priorities are expressed. How would you compare this issue to the Flock backlash, for example?
I think most people draw a strong distinction between privacy violations that are done "to them" and privacy issues with things they've chosen to use, even if they're quite similar from a technological perspective. I'd bet a lot of anti-Flock people have location sharing enabled on their phone.
Most consumers absolutely care, why do you think the biggest bipartisan issue is a massive national backlash against tech companies and a clear majority of workers being against LLM tools in the workplace?
These types of comments just show what a massive bubble you're in.
That backlash is based on conspiracy theories about data centers using up all the water and the legitimate fear of AI taking their jobs, not anything to do with privacy.
Big Tech's head is at "being evil is just being competent", "democracy is incompatible with freedom" (for me to do what I want). It is also at "humans are obsolete" point. And while Zuckenberg specifically does not have cool quotes, he was at the "who cares about genocide" and "let show weight loss ads to teenagers who we determined have low self-esteem" points in the past.
Notes: actual quote is "If you're evil, you're at least competent. And if you're evil, you're not bad. And therefore, maybe you're actually kind of good because you're at least getting something done"
I see the value of the tool but I would never use it from Meta. Would need to be self hosted with a very structured framework and guardrails so that even my local model couldn't accidentally wire 50k to a Nigerian prince or whatever the latest email scam is.
Man, I wonder how many facebook marketplace sales will be something like "In order to make sure you're a real person, before we can meet up so I can hand over this brand new macbook for $500 I need you to send me $10 electronically. It's really just to verify your commitment". With the right prompting, I'd be entirely unsurprised if Muse will just send over the $10, and of course the seller ghosts the buyer -_-
I see everybody freaking out about unfettered payment access to one's bank accounts and I keep wondering why people don't jump to the obvious solution of simply only giving it a single purpose or limited fund credit card number from privacy.com or something.
Am I missing something with the concern about ability to constrain the blast radius?
I would call that kind of thing "a very structured framework and guardrails". You'd have to come up with entirely different solutions if you want the agent to receive payments or monitor your budget.
I’m in the same boat. After witnessing context rot and inference collapse, I do not trust any LLM with mission critical work. Not Jev, not grok, not fable, not Opus.
What extent does that happen for you? I have got very good results with self hosted qwen3.8 flash next at q4 and even with qwen3.635b on a laptop. I don't keep it running forever on every single repo, its ok to leave notes in agents.md and let it search that instead of the entire history staying in context.
The Qwen models, especially when quantized, can be really bad about just trying things and seeing what works. If you’re not watching all the tool calls you may not see it, but it’s kind of scary to watch them just bump into wrong decisions and backtrack.
They also have a bad habit of accidentally building URLs that hit Alibaba infrastructure, likely because their training environment had them use those URLs. If you haven’t watched the outgoing network requests you might be very surprised at what your Qwen agents do sometimes.
heh I got a completely stupid error from gemini about some apache configs, when I pointed it out, the llm apologised, said the line(s?) should look like this, then posted the same two lines uneditted haha
The missing piece for connecting an agent to your credit card or other financials is financial liability.
If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier
“ If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier”
Sad to see such paranoia, agents can genuinely be an agent of good and positive change.
Last night I had ChatGPT go through an old email account and surface memories that I literally forgot. People who I have fond memories of. Yea I could have spent time querying gmail and digging but the agent did it in a way that really resonated. I don't care if Sama has my emails now. I do care about connecting with my past and enjoying the memories of a time long past.
Paranoid scolds of HN want the average person to be deprived of value like this for some reason.
My email is gigabytes of garbage and a few useful bits of information that are useful and have personal relevance to me. You personally, dear reader, could probably read all my emails and I wouldn’t give a shit. I just checked my sent mails for the last few years, it’s like 90% sending absence excuses to my kid’s school. Google already has it all, so I guess I could just wait for them to get off their ass and make a useful equivalent to Grok Bot and friends.
My wife has scheduled a visit with a physician and there was a younger person in there. They legit said “haha I don’t know what it is” and used ChatGPT for diagnosis.
I have no doubt they would have no problem outsourcing everything to agents.
Personally to me this feels like another classic case of starting with the technology and figuring out where to sell it as opposed to the customer experience.
Don't worry, I'm sure eventually you'll simply be required to give one of these things access to your bank account and that decision will be taken out of your hands.
Sorry, no. This is not the future I wanted. I do not run any agents nor ever will.
I find all this is just the next-gen privacy violation, disguised obviously as: "Oh, WOW an agent in the computer is doing all this for me". Bullshit.
It seems that never is enough with these ever thirsty companies, they keep pushing the limits and surprinsingly a lot of people do not care at all nor value the implications of having an arbitrary process running and doing pretty much whatever the agenda of their creators are.
You don't have to, there is a world of things you can do with them without giving them access to your email or bank account.
"Hermes, clone this android app to my computer, add this and this feature and fix this annoyance and then rebuild it and push it to my phone"
"Hermes summarize this YouTube podcast and push the summary and transcript to my Obsidian vault"
"Hermes check mg obsidian notes for when I last wrote a blog article about Neovim's integration with AI agents"
*Hermes, you have an API token that allows read only access to the Zabbix monitoring system, check that and then use the Proxmox API token that only allows you to do limited actions to reboot the VM that is having issues"
YouTube transcripts are increasingly if not completely blocked now. It used to be great to grab content from long tops. I did not have time to watch but wanted to learn about. Now. The only way to do it is to manually copy and paste the transcript information or use computer use. But I cannot easily do it on my VPS it seems unless I'm missing a trick.
It's very simple. There is ZERO chance the worlds biggest advertising companies will refrain, long-term, from using your most intimate secrets, gathered through your many conversations with their AI personal assistants, to sell you things.
"That sounds like a tough problem in your relationship right now. Would you like me to order another Crave cookie for you? There's a special right now on free delivery. What about renewing your subscription to aitherapynow.com?"
I agree with this but also think it is a first order consequence.
This level of unfettered access to your personality, lifestyle, and secrets allows for an unprecedented kind of manipulation and control. You could see it as a new kind of wealth transfer: not only will They sell you things, They will subtly manipulate behaviors of the masses via trusted agents.
Brother the biggest ad companies already own the platforms most people use for their communications. For over half the world they are the browser, the OS, the TV, the messaging platform, the map, etc.
Are all of these just flavors of "the agent has the same permissions as the user"? Not that I want to trust Meta with anything, but I'm guessing it asked for "root" access and the users granted it...?
Yes, it is clearly designed to give you access to all of this intentionally. Its system prompt (which you can just read in the interface without asking) explicitly instructs it to act on behalf of the user, not meta. All of its memory files, skills, db schema, memory integration system etc are likewise visible because they went out of their way to add an interface for viewing them!
I don't like or trust meta any more than I always did but I don't see how they could have done a "better" job with this. These kinds of agents are inherently pretty risky IMO but I don't see how this one is particularly more than any others.
Playing around with it I really don't get the sense there's any hidden prompt contradicting what's visible. It's nearly gleeful at using the VM in ways that were not intended and likely against meta's interests. I feel like someone must have won a really interesting internal power struggle to get this thing out in this form.
Probably. But also the agents are finding flaws in the security model.
Also Meta is actively encouraging users to grant them full permission, insisting with all their marketing might that it’s safe, which they know is a complete lie. Hard to blame the user when they’re being actively deceived like this. Other agent companies are more reserved and say things like “be careful” but Meta is the opposite.
> Meta’s new general-purpose AI agent Muse sent him an unsolicited notification referencing a thread between him and a co-worker over Apple Messages. Aten said he never granted Muse permissions to read his messages
Can someone explain how this is possible? If an app can do this without Full Disk Access or a popup of some kind that's a way bigger lapse on Apple's part than Meta's.
Reading the linked articles suggests that this is not possible, but Apple did acknowledge that the full-disk access grants you access to other apps (until the most recent update)
Typical "leopards ate my face" moment. You give AI (from Meta, nonetheless) full-disk access and then complain that—wow—it really meant FULL.
In a perfect world where you got nothing to hide, where companies don't sell your data and there are no hackers, even I would love to just hand all my data to Muse and have it be my trusted assistant. People who think we live in such a world are already doing that, evidently.
I saw a theory, just a theory, that it may have accessed the message via its notification preview, which I think was similarly used to leak Signal messages recently?
if you go on meta.com there is not a single mention of Facebook or Instagram anywhere in sight, and those are their leading products and money makers...
Every social media & AI company is also a surveillance company. Targeted advertising doesn't work w/o mountains of behavioral data aggregated across all of Meta's & Google's software "products".
It's always very intentional, especially with Meta/Facebook.
That's their whole modus operandi, a privacy nightmare which will feed their infinite money machine.
Reading "Careless People" didn't make me surprised at all on how the company operates, it surprised me with the personal descriptions of how people like Mark Zuckerberg and Sherryl Sandberg actually are as humans. It scared me how most people at that level of wealth and power isn't too different from the accounts in the book, they're all seriously deranged people with a gigantic lever to impose their distorted realities upon the rest of humanity.
And it's all by design, Meta and its founder have a long history of releasing products with security "flaws" that are immediately used by them to collect vast amounts of information about their victims.
Point by point:
> “OMG you can jailbreak it and get it to spill its VM”
This is the whole point; any content on the VM is yours. It runs an in isolated sandbox VM separate from stored credentials etc; this is effectively your own computer. You don’t have to trick it.
> It collects dossiers on your contacts
These are more text files that live on your private VM, alongside memory.md, etc. Do you want your secretary to forget every person you contact every day?
> It accessed Messages without full disk access
This whole story never made sense or was substantiated. Full disk access is an OS level security boundary; the user had to switch this on.
> It sold some guys stuff for too cheap and gave out his address
OK this one I basically believe, haha. Because this is the problem with Muse: the LLM is just too dumb to perform complex tasks effectively in many cases.
I just do not trust any of them, not with my money or with access to my conversations.
I could see shopping being the same way, let the agent shop for you or identify good deals on things you want but then push them to you to make the actual purchase. Like you, I'm certainly not going to let an agent make a real purchase for me unattended and I doubt any merchant is going to have much sympathy for "my AI bought this by mistake".
Unless these AI assistants are running on self-managed platforms independent of their developers, all I see is immense counterparty risk.
Comments like this are in a different reality than most consumers. Most consumers don’t care about things like avoiding lock in to a platform. If the platform solves their problem then they don’t have any reason to leave it anyway. They’re not worried if their data is used to show them more targeted ads.
Topics like this show a sharp divergence between what you read on Hacker News and how actual users operate. We already knew that people who don’t trust Facebook aren’t going to suddenly start using Muse. They were never going to consider it. For the people who do actually use Meta products, which is a customer base counted in the billions, many will not have any problem using these tools.
Or when they are hit by any other privacy fuckup like that.
[1] https://futurism.com/facebook-beauty-targeted-ads [2] https://www.bbc.com/news/health-61320202 [3] https://tech.yahoo.com/general/articles/facebook-made-money-...
These types of comments just show what a massive bubble you're in.
Notes: actual quote is "If you're evil, you're at least competent. And if you're evil, you're not bad. And therefore, maybe you're actually kind of good because you're at least getting something done"
HN must understand that they are not a representative sample of anything.
Am I missing something with the concern about ability to constrain the blast radius?
They also have a bad habit of accidentally building URLs that hit Alibaba infrastructure, likely because their training environment had them use those URLs. If you haven’t watched the outgoing network requests you might be very surprised at what your Qwen agents do sometimes.
If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier
lol… talk about delusion.
Last night I had ChatGPT go through an old email account and surface memories that I literally forgot. People who I have fond memories of. Yea I could have spent time querying gmail and digging but the agent did it in a way that really resonated. I don't care if Sama has my emails now. I do care about connecting with my past and enjoying the memories of a time long past.
Paranoid scolds of HN want the average person to be deprived of value like this for some reason.
I am perfectly prepared to believe you had a nice interaction with the robot. But this is an insane thing to say!
I have no doubt they would have no problem outsourcing everything to agents.
It feels and seems too forced.
I find all this is just the next-gen privacy violation, disguised obviously as: "Oh, WOW an agent in the computer is doing all this for me". Bullshit.
It seems that never is enough with these ever thirsty companies, they keep pushing the limits and surprinsingly a lot of people do not care at all nor value the implications of having an arbitrary process running and doing pretty much whatever the agenda of their creators are.
"Hermes, clone this android app to my computer, add this and this feature and fix this annoyance and then rebuild it and push it to my phone"
"Hermes summarize this YouTube podcast and push the summary and transcript to my Obsidian vault"
"Hermes check mg obsidian notes for when I last wrote a blog article about Neovim's integration with AI agents"
*Hermes, you have an API token that allows read only access to the Zabbix monitoring system, check that and then use the Proxmox API token that only allows you to do limited actions to reboot the VM that is having issues"
Etc, etc,
This level of unfettered access to your personality, lifestyle, and secrets allows for an unprecedented kind of manipulation and control. You could see it as a new kind of wealth transfer: not only will They sell you things, They will subtly manipulate behaviors of the masses via trusted agents.
I don't like or trust meta any more than I always did but I don't see how they could have done a "better" job with this. These kinds of agents are inherently pretty risky IMO but I don't see how this one is particularly more than any others.
Playing around with it I really don't get the sense there's any hidden prompt contradicting what's visible. It's nearly gleeful at using the VM in ways that were not intended and likely against meta's interests. I feel like someone must have won a really interesting internal power struggle to get this thing out in this form.
Also Meta is actively encouraging users to grant them full permission, insisting with all their marketing might that it’s safe, which they know is a complete lie. Hard to blame the user when they’re being actively deceived like this. Other agent companies are more reserved and say things like “be careful” but Meta is the opposite.
Can someone explain how this is possible? If an app can do this without Full Disk Access or a popup of some kind that's a way bigger lapse on Apple's part than Meta's.
Typical "leopards ate my face" moment. You give AI (from Meta, nonetheless) full-disk access and then complain that—wow—it really meant FULL.
In a perfect world where you got nothing to hide, where companies don't sell your data and there are no hackers, even I would love to just hand all my data to Muse and have it be my trusted assistant. People who think we live in such a world are already doing that, evidently.
Why are we depending on LLM "reasoning" to negotiate a price rather than just having the user enter a lowest acceptable price deterministically?
Trying to think of a number that I would consider it, and honestly $1k/month wouldn't convince me unless
- I have root on device
- device is on its own vlan, fully segmented
- !(SIM || 5G antenna)
- no google/apple id authenticated on device
- terminate agreement at any time without cost
I'm probably forgetting/not aware of ten things I should consider.
the company's brands are toxic and they know it.
That's their whole modus operandi, a privacy nightmare which will feed their infinite money machine.
Reading "Careless People" didn't make me surprised at all on how the company operates, it surprised me with the personal descriptions of how people like Mark Zuckerberg and Sherryl Sandberg actually are as humans. It scared me how most people at that level of wealth and power isn't too different from the accounts in the book, they're all seriously deranged people with a gigantic lever to impose their distorted realities upon the rest of humanity.
Facebook is like Microsoft at this point: The thing your grandparents use.
Even if they make something technically superior for a while, like what the Zune was to the iPad, tHey'll never really be cool.
and they'll certainly never be trusted.
The question implies you already are. But yes, obviously.
Related:
Updates to Full Disk Access in macOS
https://news.ycombinator.com/item?id=49937631
Meta’s Muse has a serious 0-day
https://news.ycombinator.com/item?id=49802030
Unsurprisingly, Meta's new Muse AI agent blatantly ignores users permissions
https://news.ycombinator.com/item?id=49893709
Maybe don't let Muse run your Facebook Marketplace account
https://news.ycombinator.com/item?id=49875006
What Meta got right with Muse
https://news.ycombinator.com/item?id=49946526
Muse – Meta’s personal AI agent
https://news.ycombinator.com/item?id=49615537
-Mark Zuckerberg
Today’s version of the platform targets just the right emotions to keep users “engaged.”