Integrate Varnish Cache With Magento

 Description: Here I have explained, How to install and Integrate Varnish Cache with Magento

Install Varnish Cache: 

Explore all the version of varnish in below URL

https://packagecloud.io/varnishcache

Select the require version, it provide script to install as per OS details 

# apt-get install apt-transport-https

# curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -

# echo "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1" \ 

>      >> /etc/apt/sources.list.d/varnish-cache.list

# apt update

# apt-get install varnish

# varnishd -V

Change Default Port of Web server:

Here, I am split my configuration in two file, One file for site configuration and another proxy configuration file for varnish. Change default web server from 80 to 8080. 

Note: Before integrate varnish, Change the Magento mode to Developer mode from production using below command

$ php bin/magento deploy:mode:set developer

Files:
magentotestlab.conf -- Site configuration file

upstream fastcgi_backend {
        server  unix:/run/php/php7.3-fpm.sock;
}

server {
    listen 8080;
    listen [::]:8080;
    server_name magentotestlab.com;

        access_log /var/log/nginx/magentotestlab-access.log;
        error_log /var/log/nginx/magentotestlab-error.log;


        set $MAGE_ROOT /var/www/magento2;
        set $MAGE_MODE developer;
        include /var/www/magento2/nginx.conf.sample;

}

web-proxy.conf -- Site Proxy configuration file in which we define redirect request to varnish cache

server {
         listen 80 reuseport;
         server_name magentotestlab.com;
        return 301 https://$server_name$request_uri;
}
server{
        listen 443 ssl http2;
        server_name magentotestlab.com;

       ssl_certificate /home/ssl/magentotestlab.crt;
        ssl_certificate_key /home/ssl/magentotestlab.key;

      access_log /var/log/nginx/www.kingstonkrafts.com-access.log;
        error_log  /var/log/nginx/www.kingstonkrafts.com-error.log notice;


         location / {
                 proxy_pass http://127.0.0.1:6081; #To Varnish
#                 proxy_pass http://127.0.0.1:8080; #To Vhost
                 proxy_set_header Host $host;
                 proxy_set_header X-Forwarded-Host $http_host;
                 proxy_set_header X-Real-IP $remote_addr;
                 proxy_http_version 1.1;
                 proxy_set_header Connection "";
                 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                 proxy_set_header X-Forwarded-Proto https;
                 proxy_set_header X-Forwarded-Port 443;
                 }

 }

Integrate Varnish in Magento 

Open Magento Admin and navigate to Stores -> Configuration -> Advanced -> System -> Full Page Cache and change the as follow and save config

After filled all the details download VCL for respective version [Like We have implement Varnish 6] 

Backup the existing configuration file and upload it to /etc/varnish/default.vcl

After Upload default.vcl change the path for health_check.php  as follow

  import std;
# The minimal Varnish version is 6.0
# For SSL offloading, pass the following header in your proxy server or load balancer: 'X-Forwarded-Proto: https'

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .first_byte_timeout = 600s;
    .probe = {
        .url = "/health_check.php";
        .timeout = 2s;
        .interval = 5s;
        .window = 10;
        .threshold = 5;
   }
}


### Add IP Forwarding above 2nd health_check.php define line

# IP FORWARDING
if (req.restarts == 0) {
if (req.http.X-Forwarded-For) {
std.collect(req.http.x-forwarded-for);
set req.http.X-Forwarded-For = regsub(req.http.X-Forwarded-For, "^([^,]+),?.*$", "\1");
} else {
set req.http.X-Forwarded-For = 1;
}
}

Open varnish service configuration and add below configuration to optimise varnish service configuration Comment existing ExecStart line and add below configuration

# vi /etc/systemd/system/multi-user.target.wants/varnish.service

  ExecStart=/usr/sbin/varnishd \
          -a :6081 \
          -a localhost:8443,PROXY \
          -p feature=+http2 \
          -f /etc/varnish/default.vcl \
          -p thread_pool_add_delay=2 \
          -p thread_pools=2 \
          -p thread_pool_min=400 \
          -p thread_pool_max=4000 \
          -p http_resp_hdr_len=131072 \
          -p http_resp_size=163840 \
          -p nuke_limit=1000 \
          -p workspace_client=256k \
          -p workspace_backend=256k \
          -s malloc,512m
ExecReload=/usr/sbin/varnishreload

After add above configuration reload the daemon 

# systemctl daemon-reload

After change the configuration from Magento Frontend also changed using CLI using below command

php bin/magento setup:config:set --http-cache-hosts=127.0.0.1:6081
php bin/magento config:show --scope=default --scope-code=0 system/full_page_cache/caching_application

Verify the out put from above command it should be 2. If not you can change it using below command

php bin/magento config:set --scope=default --scope-code=0 system/full_page_cache/caching_application 2

Restart Nginx service after change 

# systemctl restart nginx

Verification: To verify the cache browse the URL, Inspect the page and navigate to Network. Under Network click on any page 

You will get "x-magento-cache-debug : HIT" 

Also you can verify by make sure /magento_root/var/page_cache directory is empty. Remove the files from the path and then tried to access page again. If no file generated then it successfully integrated.