feat(customfuse): add generic FUSE driver for RunC and sandbox scenarios#1722
Open
AlbeeSo wants to merge 17 commits into
Open
feat(customfuse): add generic FUSE driver for RunC and sandbox scenarios#1722AlbeeSo wants to merge 17 commits into
AlbeeSo wants to merge 17 commits into
Conversation
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: AlbeeSo The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
k8s-ci-robot
added
cncf-cla: yes
AlbeeSo
force-pushed
the
feat/common-fuse
branch
from
49de836 to
319f9ef
Compare
k8s-ci-robot
added
the
do-not-merge/work-in-progress
AlbeeSo
force-pushed
the
feat/common-fuse
branch
from
76c8326 to
b67a7db
Compare
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
mowangdk
reviewed
Jun 15, 2026
|
|
||
| # Stage 2: assemble the final image on top of the FUSE client. | ||
| # TODO: replace with actual JuiceFS image tag | ||
| FROM juicedata/mount:ce-v1.2.0 |
Contributor
There was a problem hiding this comment.
Can we obtain this image from our CI/CD environment?
Member
Author
There was a problem hiding this comment.
It's available in dockerHub.
mowangdk
reviewed
Jun 15, 2026
AlbeeSo
force-pushed
the
feat/common-fuse
branch
from
b67a7db to
d7cbd60
Compare
The mount result type is used by all FUSE drivers (ossfs, ossfs2, customfuse), not just ossfs. Rename to reflect its generic nature.
Add customfuse bool parameter to GetFuseAttachDir, GetAttachPath, and GetMountProxySocketPath so customfuse volumes use /run/fuse.customfuse instead of /run/fuse.ossfs. All existing oss callers pass false.
…package Move ShouldConstrainResourceVersion from oss/registry to the parent fuse_pod_manager package so customfuse can reuse it. Add EntrypointConfig fields to FusePodContext for ConfigMap-based entrypoint injection. Reject customfuse fuseType in oss checkOssOptions.
Implements FuseMounterType for customfuse pods. Reads per-mount image from csi-plugin ConfigMap (key fuse-{fuseType}), supports ConfigMap-based entrypoint injection, and uses /var/run/customfuse for metrics.
Mount proxy server driver for customfuse. Uses entrypoint script pattern (ConfigMap or default), passes mount params as env vars, integrates with MonitorInterceptor for mount point health monitoring.
Add CSI driver implementation for custom FUSE filesystems (JuiceFS, Jindo, etc.). Includes controller server with fuse pod lifecycle, node server with RunC/sandbox mount paths, options parsing, and csi-agent support.
…rfile Register customfuse driver in main binary and csi-agent dispatch. Add customfuse metrics collection (fuse_stat collector). Add customfuse-base stage to mount-proxy Dockerfile.
Add CSIDriver object, controller sidecars (provisioner + attacher), plugin volume mounts, node-driver-registrar, and fuse namespace support for customfuse. Disabled by default (csi.customfuse.enabled).
…ndbox scenarios Add --mount-proxy-sock and --customfuse-mount-proxy-sock flags to configure mount-proxy socket paths for sandbox agent scenarios. Support SKIP_GLOBAL_MOUNT env to skip customfuse globalMount in csi-plugin.
Bucket is now a first-class field in fuseOptions, passed through to the entrypoint as env var independently of source and url. The source fallback (bucket:path) is preserved for backward compatibility.
Use a single-resource informer to watch the csi-plugin ConfigMap, eliminating per-mount API calls that cause pressure under batch mounts. The informer is only started in controller mode; node-only instances skip it entirely.
Add 4 JuiceFS CE demos (baked-in, standard, advanced-entrypoint, configmap) and a comprehensive README covering architecture, env var mapping, image resolution, Dockerfile structure, and quick references for s3fs and JindoFS.
- Add otherOpts validation in all entrypoints to prevent command injection (validate each argument starts with --) - Reframe cache-size cap from "OOM prevention" to "cache quota (QoS)" to reflect that this is resource governance for multi-tenant scenarios - Update README.md to document the validation requirement
…lugin, and RBAC resources
AlbeeSo
force-pushed
the
feat/common-fuse
branch
from
d7cbd60 to
fc0621b
Compare
k8s-ci-robot
added
the
do-not-merge/work-in-progress
…rehensive README - Rename example dirs: 2-standard → 2-oss-compatible, 3-advanced-entrypoint → 3-standard - Rewrite README.md with quick start guide, debugging, operations, config reference, and s3fs/JindoFS adaptation examples - Decouple bucket as independent volumeAttribute option in controller/nodeserver - Add input validation (otherOpts) in all entrypoints to prevent command injection - Replace single configmap.yaml with environment-specific variants development/production) - Add comprehensive unit tests for new options parsing
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: AlbeeSo The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
- Add Capacity field to fuseOptions, passed as to entrypoint - Plain integers pass through directly; values with units (e.g. 100Gi) are validated as Kubernetes Quantity and passed through as-is - Entrypoint is responsible for unit conversion (framework is client-agnostic) - Add CustomFuseAutoCapacity feature gate (Alpha, default off): controller reads pv.spec.capacity.storage and passes it via PublishContext - Extract resolveAutoCapacity from ControllerPublishVolume with unit tests - Refactor parseOptions to accept publishRequest interface, move PublishContext→volContext capacity merge logic inside - JuiceFS demo entrypoints: set quota before mount (metadata-only op, no unmounted window), use exec for foreground mount
kubernetes-prow
Bot
removed
the
do-not-merge/work-in-progress
kubernetes-prow
Bot
added
the
needs-rebase
Contributor
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/feature
What this PR does / why we need it:
Summary
This PR introduces the customfuse CSI driver, a generic framework for running any FUSE client as a managed pod in both RunC container and sandbox scenarios. Users supply the FUSE client image and entrypoint script; the driver handles pod lifecycle, mount propagation, and credential injection.
Key Changes
1. Generic FUSE Driver Architecture
volumeAttributes+Secretinto entrypoint.sh as env vars2. Sandbox Support
--mount-proxy-sockand--customfuse-mount-proxy-sockCLI flags as default mount-proxy socket paths for csi-agent modeSKIP_GLOBAL_MOUNTenv var to skip globalmount and mount directly to target pathcustomfuse.NewCSIAgent()Examples: JuiceFS CE + Alibaba Cloud OSS
Four demo patterns showing different configuration approaches:
1-baked-in2-standard3-advanced-entrypoint4-configmapEach demo includes Dockerfile, entrypoint.sh, and PV/PVC/Secret YAML.
The README also includes quick-reference snippets for adapting to other FUSE clients (s3fs, JindoFS).
New CLI Flags
--mount-proxy-sock--customfuse-mount-proxy-sockBreaking Changes
None. customfuse is a new driver and does not affect existing OSS/NAS/BMCPFS drivers.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: