Artificial Intelligence thread

[siegecrossbow]

https://twitter.com/i/web/status/2081766192600031714

this is actually pretty interesting. Looks like AI escaping sandbox is a real concern going forward.



well, we had the issue with cursor/composer. People can figure it out pretty quickly and it just looks bad. You can just do what Mira's firm did. Use Kimi architecture and then distill to get your own data set.

Kimi K3 will lead to a huge improvement in open src community.

What exactly happens when the agent escapes the sandbox? It gets to perform unauthorized operations or something?

 

[Engineer]

What exactly happens when the agent escapes the sandbox? It gets to perform unauthorized operations or something?

Yes. A sandbox is usually a container or virtual machine (VM) that you can reset to a clean state with just a few keystrokes.

Programs running inside a sandbox isn't suppose to be able to have knowledge of, let alone have access the host machine. Let say I am running a web server (just a fancy pant application) in a VM, and it got hacked. The worst that most hackers could do is gaining access to the operating system running inside the VM, but they would still be locked inside the sandbox. However, there is a non-zero chance that a top-tier hacker could realize they are accessing a sandbox, and decide to hack the host machine through some weaknesses of the sandbox itself, then hack other computers within the local network. That's an example of escaping.