Our Life Together  >  Thoughts  >  Adding Brotli to an Already-built NGINX Instance  —  { code }

Do you have a running instance of NGINX and wanted to add Brotli compression? What if you don’t want to recompile a custom web server from source or replace your auto-updating repo-installed version? You can create dynamic Brotli modules which upgrade whenever NGINX does.

The bash script below explains the installation steps. Feel free to study and execute the codes step-by-step. That may help you troubleshoot any peculiarities in your system. Otherwise, simply run the script.

/usr/local/sbin/mkbrotli

You may choose to automate the installation by simply running the script above. In its default settings, it will process the version number you will pass to it. To get your NGINX’s version number run this command:


nginx -v
nginx version: nginx/1.17.1

Since the version number in this example is 1.17.1, after you have downloaded the bash script above, go to the directory where the script is and run these commands:


chmod +x mkbrotli
./mkbrotli 1.17.1

You may now chill and relax as the script automagically downloads, builds, and installs the dynamic modules for you. If the script runs through errors please let me know in the comments below so I may make adjustments to it.

Activating the modules

Now that we have the modules, there are two configuration steps needed to activate Brotli compression. First, load the modules in NGINX’s main configuration file (mine was at /etc/nginx/nginx.conf). You can get your config file path by running this command:


nginx -V 2>&1 | grep -o 'conf-path=[^ ]*' | sed 's/conf-path=//'

Now edit NGINX’s conf file. Outside any block { ... }, maybe on the top section of the file, add these lines:

nginx/nginx.conf

load_module modules/ngx_http_brotli_filter_module.so;
load_module modules/ngx_http_brotli_static_module.so;

# http {
#   ...
# }

Second, activate Brotli (and set its configuration directives, aka settings) either inside the http block of NGINX’s conf file, or the server block of your domain configs (e.g. files in nginx/sites-enabled/). As for me, I placed these directives in the nginx/snippets/ folder and called them via include in my domain configs.

nginx/nginx.conf
Auto-update with NGINX

One potential problem with dynamic modules is when NGINX automatically upgrades via a package manager (e.g. apt, yum, or pacman). If you’re like me, all upgrades in my server are automated by scripts. You see, dynamic modules are version-specific. NGINX is set not to load modules that were built for a different version. So how will Brotli modules automatically upgrade together with NGINX?

The answer is to hook on the package manager during upgrades. In apt this may be a two- or three-step process:

  1. Hook to apt and call a worker script. (Alternatively, the hook could directly call mkbrotli and thus skip the need for a worker script.)
  2. The worker script calls one or more build scripts to update the dynamic modules.
  3. Build scripts will build and install the new modules right before NGINX upgrades.

Here is my hook file in /etc/apt/apt.conf.d/:

/etc/apt/apt.conf.d/05nginxmodules

The worker script I use:

/usr/local/sbin/nginx-mod-preinstall

And finally the build script for this one is… (drum roll please)… of course mkbrotli. 😉 I hope that Brotli could be natively supported by NGINX in the future. It’s awesome. For now, we have this workaround. Enjoy the compression!

Google PageSpeed

Another project that aims to speed things up on the web is Google’s best-practices optimization called PageSpeed. I also made a Bash script to automate installation of PageSpeed on a running NGINX server. If you’re interested, check out this post: Adding PageSpeed to an Already-running NGINX Instance.

</>

Categories: Thoughts

Majal Mirasol

Servant of the Creator happily married to Jehovah’s slave girl.

While Majal was absorbed in training for a good foundation in the Maths and Sciences, he learned that there is more to life than simply elucidating complex designs in our universe. He realized that there is a Designer. The Source of our order, an order much superior than what our human brains could conceive.

14 Comments

Nigel · July 27, 2019 at 11:18 pm

I get this error on nginx test

module “/usr/share/nginx/modules/ngx_http_brotli_filter_module.so” is not binary compatible in /etc/nginx/nginx.conf

Any ideas?

    Majal Mirasol · July 28, 2019 at 5:49 pm

    Hi Nigel! We should supply to the script the same version of NGINX we are running. Have you checked via nginx -v? The warning will also show the version number the module was built for, and your running NGINX’s version number. Can you please post the full warning message here along with the mismatched version numbers?

      Lowi · August 8, 2019 at 10:13 pm

      Hi, thanks for the script. I get the same error and I’m sure I installed the correct version


      Sucessfully built Brotli for NGINX 1.14.0 in /usr/lib/nginx/modules

      ####################################################################

      $ nginx -t
      nginx: [emerg] module “/usr/share/nginx/modules/ngx_http_brotli_filter_module.so” is not binary compatible in /etc/nginx/nginx.conf test failed
      $ nginx -v
      nginx version: nginx/1.14.0 (Ubuntu)

        Majal Mirasol · August 8, 2019 at 10:21 pm

        That’s interesting… I’m pretty sure it works in my system. In fact it is running on this very website’s server. Hmmm… I’ll try to look more into this. Can you please supply more relevant details of your system? It may help us pinpoint and troubleshoot this issue. Thanks for the help!

        Looking again, I’d just like to confirm also that your test command should be: $ sudo nginx -t rather than $ nginx -t?

        Just guessing. Or the problem may be deeper than this… Thank you for your inputs! 🙂

          Scott · August 31, 2019 at 3:00 pm

          I got the same issue. It’s because you need to configure the module with the same options as nginx -V in the system (which is a lot when using debian bionic for example) – once I updated the script to contain the extra config options it worked

            Majal Mirasol · August 31, 2019 at 3:42 pm

            Hi Scott! Nice find. The configure step flag --with-compat is supposedly designed to handle that. But I think I read somewhere that it may not also work with certain configurations of NGINX. Thanks for pinning this. I’ll soon update the script which will include flags from nginx -V. Is it also okay if you could post how your configure command looked like? I and the readers here really appreciate all the help! 🙂

            Majal Mirasol · September 3, 2019 at 12:00 am

            Hello Scott! I’ve just updated the script. Please find new section confparams in it. 🙂 Thanks for your help!

        Majal Mirasol · September 2, 2019 at 11:55 pm

        Hi Lowi! A comment by Scott identified the culprit: your NGINX was likely configured without the --with-compat flag. I’ve updated the script where you can now choose to load the configure parameters of the installed NGINX in your system. Please see section on confparams and uncomment one of the commands that will work for you. Have a good day!

          Remco · September 12, 2019 at 8:55 pm

          Hi Majal, i keep getting the binary not compatible error, even with uncommenting one of the 3 confparams you provided in the script.

          nginx version: nginx/1.14.0 (Ubuntu)
          built with OpenSSL 1.1.1 11 Sep 2018 (running with OpenSSL 1.1.1c 28 May 2019)
          TLS SNI support enabled
          configure arguments: –with-cc-opt=’-g -O2 -fdebug-prefix-map=/build/nginx-DUghaW/nginx-1.14.0=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2′ –with-ld-opt=’-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -fPIC’ –prefix=/usr/share/nginx –conf-path=/etc/nginx/nginx.conf –http-log-path=/var/log/nginx/access.log –error-log-path=/var/log/nginx/error.log –lock-path=/var/lock/nginx.lock –pid-path=/run/nginx.pid –modules-path=/usr/lib/nginx/modules –http-client-body-temp-path=/var/lib/nginx/body –http-fastcgi-temp-path=/var/lib/nginx/fastcgi –http-proxy-temp-path=/var/lib/nginx/proxy –http-scgi-temp-path=/var/lib/nginx/scgi –http-uwsgi-temp-path=/var/lib/nginx/uwsgi –with-debug –with-pcre-jit –with-http_ssl_module –with-http_stub_status_module –with-http_realip_module –with-http_auth_request_module –with-http_v2_module –with-http_dav_module –with-http_slice_module –with-threads –with-http_addition_module –with-http_geoip_module=dynamic –with-http_gunzip_module –with-http_gzip_static_module –with-http_image_filter_module=dynamic –with-http_sub_module –with-http_xslt_module=dynamic –with-stream=dynamic –with-stream_ssl_module –with-mail=dynamic –with-mail_ssl_module

          You got any idea what i can do?
          104990#104990: module “/usr/lib/nginx/modules/ngx_http_brotli_static_module.so” is not binary compatible in /etc/nginx/nginx.conf

            Majal Mirasol · September 15, 2019 at 8:38 pm

            Hi Remco! It appears that the source of this issue is because NGINX could be just be configured in so many different ways. My NGINX comes vanilla from Ubuntu’s repos and works out of the box without even using the confparams part of the script. It seems that you have a customized build of NGINX. You may likely troubleshoot your issue by playing with the values of confparams. How about feeding confparams directly with your configuration parameters starting from --prefix= up to the end of the line? What I also usually do with bash scripts is to run the commands step-by-step in a terminal and manually check for errors. I’m sorry but I cannot test your issue since we are using different configurations of NGINX. We will appreciate it if you could post back your findings about your specific setup. It will be helpful to other readers who may also be using custom NGINX builds. Enjoy your day!

Igor «InoY» Z. · August 8, 2019 at 7:45 pm

Thank you! I do not try this manual yet but it is exactly what I’ve been looking for!

Also I have a question: can I translate this manual to Russian? Of course with mention to you and links to your blog?

    Majal Mirasol · August 8, 2019 at 9:28 pm

    You’re welcome to do so Igor. I haven’t completed the Terms of Use for the site yet, but I’m planning that while others aspects of the site remains in copyright, the codes be released under GNU General Public License v3.0 copyleft. I love open source, and I’m happy to give back small contributions to make our world a better place. 🙂

SeCrEt BoY™ · September 30, 2019 at 2:18 am

Hook to apt and call a worker script. (Alternatively, the hook could directly call mkbrotli and thus skip the need for a worker script.)
The worker script calls one or more build scripts to update the dynamic modules.
Build scripts will build and install the new modules right before NGINX upgrades.

I have copied 2 your files to /etc/apt/apt.conf.d/05nginxmodules and /usr/local/sbin/nginx-mod-preinstall
And what do I need to do next sir?
And finally the build script for this one is… (drum roll please)… of course mkbrotli. >>> how to build?
Thanks for help!

    Majal Mirasol · September 30, 2019 at 6:03 am

    Hi SeCrEt BoY! If you copied those two files in their proper directories, and if mkbrotli is placed in /usr/local/sbin/ then all you have to do is wait until NGINX upgrades and check if the apt hook works. Of course you will need to confirm first if the mkbrotli script is working well with your system, that is, if it runs and installs without issues.

    On your other question if I understood correctly, did you mean how to run mkbrotli and make it build Brotli? After you copied mkbrotli to /usr/local/sbin/ and made sure it is executable, just type this in the terminal: $ /usr/local/sbin/mkbrotli and see whether it will be successful or not.

    I hope this helps.

Leave a comment