Add elgentos/magento2-varnish-extended dependecy for optimized varnish VCL generation

[dadolun95]

As discussed during Mage-OS hackaton on December 14, we should include optimized vcl version for varnish.
This dependency includes bf-cache support, removes all marketing get parameters to minimize the cache objects, adds vmod xkey support.

[rhoerr]

Thanks! I agree with this in principle. I want to review the module (but I know it's widely used and well trusted). I'm in favor of this for Mage-OS 3.0.

[marcelmtz]

I'm taking a look on this one. Ideally I'll be running some manual test, run automated tests, but any comment/idea to improve the testing is welcomed @dadolun95 @rhoerr

[rhoerr]

Note the module requires PHP 8.3+, but Magento 2.4.9 (and by proxy Mage-OS 3) will too, so that's fine.

[marcelmtz]

Few information
Store that gets this module installed without configuring VCL (active immediately after install) :

• PHP side:
  • PurgeCache replacement — this is the only class that runs at runtime during normal store operation. It becomes active the moment the module is installed. The key change is the tag chunking logic. However, it's fully backward-compatible: it still sends PURGE requests with the same X-Magento-Tags-Pattern header the old VCL already understands. The old VCL handles them identically via ban(). This is actually a transparent improvement.
  • Soft purging is 0 by default (confirmed in config.xml:119), so no X-Magento-Purge-Soft header is sent. Even if someone enables it, the old VCL simply ignores the unknown header and does a regular ban — no breakage.
  • xkey is 0 by default (config.xml:118). Even if enabled, xkey is entirely a VCL-side feature — the PHP side still sends regular PURGE requests. No breakage.
  • Admin observers (marketing parameter warnings) are purely UI notifications. No impact on runtime behavior.
    VCL side (inert until you regenerate and reload)
• Purging/Caching
  • I did some manual tests, looks like invalidating, purging, caching works pretty good

Store that gets this module installed with the VCL configured

• Purging/Caching
  • I did some manual tests, looks like invalidating, purging, caching works pretty good
• Able to place orders

PHP 8.3
If MageOS 3 its paired with Magento 2.4.9 in requirements, we should be okay then as it will have 8.4 - 8.5

To sum up:
This module seems to work good with MageOS branch release/3.x ✅

[marcelmtz]

This looks correct. As this is an external repository It shouldn't require any extra config as for the previous mage os version history JSON files will get "frozen"

[marcelmtz]

@ryan
Do you have any con for getting this merged to see if I can hit the button, is there something I might be missing?

[rhoerr]

@peterjaap What do you think?

Is this module something that can be bundled with Mage-OS for general installation?

Should it be?

(Will it be beneficial? Are there risks or side effects we should consider?)

[peterjaap]

@rhoerr it definitely can! One thing to note is that the notation for variables in the VCL has changed.

Magento core uses:

.host = "/* {{ host }} */"; (see here)

Our extension uses;

.host = "{{var host}}"; (see here)

I've seen this trips some people up.

Also, in an earlier update BF cache has been added to Mage-OS, which is also in this extension. So that would then have double functionality.

[marcelmtz]

@rhoerr it definitely can! One thing to note is that the notation for variables in the VCL has changed.

Magento core uses:

.host = "/* {{ host }} */"; (see here)

Our extension uses;

.host = "{{var host}}"; (see here)

I've seen this trips some people up.

Also, in an earlier update BF cache has been added to Mage-OS, which is also in this extension. So that would then have double functionality.

I just did a quick comparison with claude

VCL Comparison: default (OOTB) vs elgentos extension

Feature	default OOTB	elgentos extension
VCL Version	4.0	4.1
Purge speed	❌ Slow ban-based	✅ Fast xkey purging
Full cache flush	❌ Expensive regex ban	✅ Instant via all tag
Soft purge	❌ Not supported	✅ Supported via xkey.softpurge
Cookie handling	⚠️ Fragile regex-based	✅ Robust vmod-based
Static/media caching	❌ Always bypassed	✅ Configurable via flags
Browser/CDN caching	❌ Disabled (no-store)	✅ Short TTL allowed (max-age=60)
Grace period visibility	✅ Explicit in headers	⚠️ Less granular
Gzip compression	✅ Built-in	❌ Not included
Complexity	✅ Simple, no extra deps	⚠️ Requires xkey vmod installed
Tracking param stripping	⚠️ Hardcoded list	✅ Configurable via template
Host header normalization	❌ Not handled	✅ Port stripping included
Empty query string cleanup	❌ No	✅ Yes
Pass-on-cookie rules	❌ Not configurable	✅ Configurable via template

I think this is really cool

@peterjaap @rhoerr Looks like the bf cache support was introduced in the theme-optimization module https://github.com/mage-os/module-theme-optimization/blob/main/Plugin/Framework/App/Response/Http.php

Here's a quick comparison between them:

Layer	mage-os/module-theme-optimization	elgentos/magento2-varnish-extended
Built-in FPC	Strips no-store via PHP plugin (Plugin/Framework/App/Response/Http.php)	Not applicable
Varnish	Documented gap — manual VCL edit required	Handles via {{if enable_bfcache}} toggle in vcl_deliver
Browser JS restore	Full pageshow handler — minicart refresh, login state check, mobile menu close	Not present
Admin UI	Full config panel (enable/disable, minicart interaction mode, URL exclusions)	Not present

IMHO The modules are complementary, not alternatives. module-theme-optimization's own README already recommends elgentos for Varnish users (feel free to correct me if anything I've added is wrong or I am hallucinating)

I can run some tests with both running, what do you think?

[rhoerr]

The theme optimization module is definitely complementary with this, not conflicting. We decided not to touch varnish configuration there at all. They should be fine to use together. Bfcache there really only applies to native FPC.

What happens if varnish xkey isn't installed? It looks like there's a setting to turn that aspect off.

[marcelmtz]

The theme optimization module is definitely complementary with this, not conflicting. We decided not to touch varnish configuration there at all. They should be fine to use together. Bfcache there really only applies to native FPC.

What happens if varnish xkey isn't installed? It looks like there's a setting to turn that aspect off.

There is an option to disable it and it will fallback to the ban mechanism
{{else}} ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern); return (synth(200, "0")); {{/if}}

The only concern is when the configuration is enabled and xkey is not installed, I think it will break Varnish compilation error and/or the service might start to behaving weird or to reload. Ideally this is covered, but a configuration could bring the site down (I assume there are others configurations that can bring the site down as well already in Magento if you don't have locked the configuration)

[peterjaap]

@marcelmtz I think the VCL won't even compile so it won't bring the site down

[rhoerr]

Thanks! Given that, I'm good with bundling this. It seems like a clear improvement for anyone that is using Varnish and applies the updated VCL, and minimal risk or impact for anyone that isn't.

Let's hold a week for any other input.

[peterjaap]

Just tested, the VCL indeed doesn't compile when xkey isn't installed.

VCL compilation failed
./vcl/default.vcl:83: Unknown vmod 'xkey'

The VCL won't load at all — it stays in a failed state and your previously active VCL remains in use (if one was already loaded). So no traffic impact if you're hot-reloading on a running instance.

[ProxiBlue]

I gather it can still be disabled if needed?

My concern woudl be automated CI deploy on initial release to hosting like hypernode.

Can this affect compatibility?
Not a major expert on varnish related things, but already seen syntax had changed, so this woudl need to be carefully deployed under attendance form developer,and not via automated CI pipeline, as it seems manual work will need to be done?

If so, this needs to be very well highlighted, and documented steps to perform maybe?
Might be worth asking our friends at hypernode on their comments/thoughts on a change like this?

Basically...how can this affect managed hosting solutions, and deployment? (all my deploys are 100% automated on a daily deploy schedule. what is in live branch, gets deployed.)

I am for it, why not improve, things

[peterjaap]

@ProxiBlue the extension doesn't do anything if you don't generate the VCL. So unless you generate the VCL in your pipeline, nothing happens after adding it.

[ProxiBlue]

@peterjaap and it does not alter or disable existing generate vcl for varnish currently setup with my hypernode. that will continue to function after this is deployed into server?

There were a few manual steps to do to setup varnish into hypernode, https://docs.hypernode.com/ecommerce-applications/magento-2/how-to-configure-varnish-for-magento-2-x.html#configure-magento-2-x-for-varnish

So I just need to be sure that won;t break, and current vcl already setup and imported will just keep on working?

[peterjaap]

Yes that will change, since the command varnish:vcl:generate is then taken care of by https://github.com/elgentos/magento2-varnish-extended/blob/master/Console/Command/GenerateVclCommand.php

It should just do a fallback using the new VCL. So it will apply the optimized VCL when you run those commands that are mentioned in the Hypernode docs.

[marcelmtz]

Just tested, the VCL indeed doesn't compile when xkey isn't installed.

VCL compilation failed
./vcl/default.vcl:83: Unknown vmod 'xkey'

The VCL won't load at all — it stays in a failed state and your previously active VCL remains in use (if one was already loaded). So no traffic impact if you're hot-reloading on a running instance.

Thank your for testing it out <3

[furan917]

Added two PRs to the elgentos repo I would love to see included before its merged into Mage-OS

Security: elgentos/magento2-varnish-extended#126
Cache poisoning with vary change: elgentos/magento2-varnish-extended#127

[rhoerr]

I poked the PRs. Going to merge this now either way.