Replace the S3 FUSE backend (mount-s3) with JuiceFS

[HaidYi]

Is your feature request related to a problem? Please describe.

Daytona Volumes are currently backed by S3-compatible object storage mounted via FUSE (e.g. Mountpoint for Amazon S3). While this works well for simple reads and writes, it lacks full POSIX semantics, which causes real-world failures for a broad class of agent workloads:

• Atomic rename fails — tools and libraries that write files safely by doing write tmp → rename to dest (e.g. JSON config writers, SQLite, many Python libraries) crash with EINVAL or EXDEV because rename(2) is not supported across S3-backed FUSE mounts.
• Random-write access fails — file formats like HDF5 (.h5ad), SQLite databases, and other binary formats require random-write and seek() support. Attempting to write these directly to a mounted Volume produces OSError: [Errno 22] Invalid argument and can cause segfaults inside native libraries (e.g. h5py).
• No file locking — flock / fcntl locks are not supported, breaking tools that rely on them for concurrency control.

These are not edge cases — they are standard POSIX behaviors that virtually every serious data processing, ML, and scientific computing workload depends on. Agents writing model checkpoints, datasets, or intermediate results to a Volume will hit these errors today.

Describe the solution you'd like

Replace (or offer as an option) the current S3 FUSE backend for Volumes with JuiceFS, an open-source, fully POSIX-compatible distributed filesystem designed exactly for this use case.

JuiceFS works by separating metadata from data:

• Data is stored in any S3-compatible object store (e.g. MinIO — which Daytona already runs internally)
• Metadata (directory tree, file attributes, rename atomicity) is stored in a fast key-value store (e.g. Redis — which Daytona also already runs internally)

This architecture gives JuiceFS full POSIX compliance, including atomic rename(2), random writes, flock, and mmap — without sacrificing the scalability and persistence of object storage.

The key insight: Daytona's existing infrastructure is already a perfect match for JuiceFS. Daytona's Helm chart already ships MinIO as the object storage backend and Redis for session/state management. JuiceFS requires exactly these two components. Enabling JuiceFS volumes would require no new infrastructure dependencies — just wiring the existing MinIO and Redis instances into a JuiceFS volume and exposing the mount point inside sandboxes.

The implementation path would be:

1. Format a JuiceFS volume using the existing MinIO (data) and Redis (metadata) instances at volume creation time
2. Mount the JuiceFS volume into sandboxes via FUSE in place of (or alongside) the current raw S3 FUSE mount
3. Expose this as a Volume option (e.g. posix: true or a new volume type) so users can opt in

Describe alternatives you've considered

• MesaFS — supports POSIX and has a Daytona integration guide, but is a third-party commercial product and is currently blocked for Tier 1/2 users due to Daytona's network restrictions on Essential Services only.
• EFS / NFS — true POSIX block storage, but not part of Daytona's current infrastructure and would require significant new dependencies.
• s3fs-fuse / goofys — simulate some POSIX operations but are known to be unreliable for rename and random writes in production; not a real fix.
• Workarounds in userspace — e.g. writing to /tmp and syncing to S3 manually. This works but forces every agent and every tool to be aware of the storage limitation, defeating the purpose of a transparent Volume abstraction.

Additional context

The two concrete failure modes that motivated this request:

1. A metadata staging step that uses the standard write tmp → mv tmp dest pattern fails with mv: cannot move: Invalid cross-device link on a mounted Volume.
2. Writing an AnnData / HDF5 file (.h5ad) directly to a mounted Volume fails with:

OSError: [Errno 22] Driver write request failed
(file write failed: errno = 22, error message = 'Invalid argument',
buf = 0x..., total write size = 35664, bytes this sub-write = 35664, offset = 44608)
Segmentation fault (core dumped)

Both failures are direct consequences of missing POSIX semantics in the current S3 FUSE backend. JuiceFS resolves both at the filesystem level, transparently, without requiring changes to agent code.

Licensing and commercial viability

JuiceFS Community Edition is released under the Apache License 2.0 — the same license used by the majority of cloud-native infrastructure projects (Kubernetes, Prometheus, etcd, etc.). This is the most permissive and commercially friendly open-source license available:

• Daytona can embed, ship, and distribute JuiceFS as part of its platform without any copyleft or open-source obligations
• No restrictions on commercial use, modification, or redistribution
• No license compatibility issues with Daytona's own AGPL-3.0 codebase when used as a linked dependency

It is worth noting that JuiceFS originally launched under AGPLv3 in January 2021, but explicitly switched to Apache 2.0 in January 2022 in direct response to commercial adoption concerns from the community. This was a deliberate signal of long-term commitment to commercial-friendly open source.

JuiceFS is production-proven at scale, used in production by companies including Shopee, Xiaomi, Baidu, DJI, SenseTime, and many others for petabyte-scale ML and data workloads. Combined with its Apache 2.0 license and natural architectural fit with Daytona's existing MinIO + Redis stack, this feels like a low-integration-cost, high-impact improvement for any agent workload that does real file I/O.