Rendez-vous sur Arrakis, Le site perso d'un hacker libriste curieux crêpophile étourdi
Le 13/09/2019 à 13:39 dans /Logiciel-libre/Code/

sbw : save bandwidth cgi for httpd

EN

sbw est un petit script cgi de 89 lignes en shell (oui, en shell, et alors?) avec aussi peu de dépendances que possible. Son objectif est de réduire l'utilisation de bande passante en servant lorsque c'est possible les fichiers gzippés. En bonus, il permet de définir une page 404 personnalisée. Il est censé s'utiliser avec OpenBSD et le serveur httpd.

Lorsqu'une personne demande la page domain.tld/index.html, sbw regarde si index.html.gz est disponible et envoie ce fichier à la place. De plus, il indique au navigateur le hash du fichier pour qu'il gère mieux la mise en cache (header ETag).

$ grep -Ev "^#|^$" sbw.cgi | wc -l
89

Configuration

Activez et démarrez slowcgi(8).

# rcctl enable slowcgi
# rcctl start slowcgi

Configurez httpd(8) pour servir les types de fichiers désirés avec sbw :

# cat /etc/httpd.conf
    [...]
    location "/*.html" { include "/etc/httpd.d/sbw.conf" }
    location "/*.css" { include "/etc/httpd.d/sbw.conf" }
    location "/*.js" { include "/etc/httpd.d/sbw.conf" }
    location "/*.svg" { include "/etc/httpd.d/sbw.conf" }
    location "/*.xml" { include "/etc/httpd.d/sbw.conf" }
    location "/*.atom" { include "/etc/httpd.d/sbw.conf" }
    location "/*.txt" { include "/etc/httpd.d/sbw.conf" }
    location "/*/" { include "/etc/httpd.d/sbw.conf" }
[...]
# cat /etc/httpd.d/sbw.conf
    root "/cgi-bin/sbw.cgi"
    fastcgi param realroot "/htdocs/domain.tld/" 
    fastcgi param cachecontrol "1814400" 
    fastcgi param file404 "/_404.html" 

Modifiez les paramètres selon vos besoins :

Installez sbw.cgi:

# ftp -o /var/www/cgi-bin/sbw.cgi https://dev.ybad.name/sbwcgi/sbw.cgi
# mkdir -p /var/www/bin
# cp /bin/cat /var/www/bin/
# cp /bin/sha256 /var/www/bin/
# cp /bin/sh /var/www/bin/

Compressez les fichiers que vous voulez servir gzippés :

$ gzip -9 < index.html > index.html.gz

Vous en avez sans doute un grand nombre, donc plutôt que de tout faire à la main, voici un script qui va gzipper tous vos fichiers textes (en gardant les originaux) s'ils sont suffisamment gros pour présenter un avantage :

#!/bin/sh
# search for files candidates for gzip
# according to extension and size
# this gzip text files in $1 dir only if newer

[[ $# -eq 1 ]] || exit 1 

find "${1}" -type f \
    -size +1500c -a \
    \( \
    -name "*\.html" -o \
    -name "*\.css" -o \
    -name "*\.js" -o \
    -name "*\.svg" -o \
    -name "*\.xml" -o \
    -name "*\.atom" -o \
    -name "*\.txt" \
    \) \
    | while read -r line; do
    echo "gzip $line"
    gzip -9 < "${line}" > "${line}.gz"
done

Lancez-le ainsi :

$ gzip_site.sh /path/to/your/website

Rechargez httpd puis testez

# rcctl restart httpd

FAQ

BUGS

Peut-être. Merci de m'en faire part.


FR

sbw is a shell (yes, shell) cgi script with the fewest requirements to save bandwidth. Used with OpenBSD httpd, you can enable gzip compression and set a custom 404 page.

When a page is requested, it tries to serve the gzipped file if it is present : if someone access to domain.tld/index.html , sbw try to return index.html.gz. It fills the ETag header to make sure the browser handle cache properly.

$ grep -Ev "^#|^$" sbw.cgi | wc -l
89

Setup

Enable and start slowcgi(8).

# rcctl enable slowcgi
# rcctl start slowcgi

Edit httpd(8) configuration for desired file extension :

# cat /etc/httpd.conf
    [...]
    location "/*.html" { include "/etc/httpd.d/sbw.conf" }
    location "/*.css" { include "/etc/httpd.d/sbw.conf" }
    location "/*.js" { include "/etc/httpd.d/sbw.conf" }
    location "/*.svg" { include "/etc/httpd.d/sbw.conf" }
    location "/*.xml" { include "/etc/httpd.d/sbw.conf" }
    location "/*.atom" { include "/etc/httpd.d/sbw.conf" }
    location "/*.txt" { include "/etc/httpd.d/sbw.conf" }
    location "/*/" { include "/etc/httpd.d/sbw.conf" }
[...]
# cat /etc/httpd.d/sbw.conf
    root "/cgi-bin/sbw.cgi"
    fastcgi param realroot "/htdocs/domain.tld/" 
    fastcgi param cachecontrol "1814400" 
    fastcgi param file404 "/_404.html" 

Edit parameters to suit your needs :

Install sbw.cgi:

# ftp -o /var/www/cgi-bin/sbw.cgi https://dev.ybad.name/sbwcgi/sbw.cgi
# mkdir -p /var/www/bin
# cp /bin/cat /var/www/bin/
# cp /bin/sha256 /var/www/bin/
# cp /bin/sh /var/www/bin/

Gzip files you want to serve gzipped:

$ gzip -9 < index.html > index.html.gz

Since you might have quite lot, here is a script to gzip all your text files :

#!/bin/sh
# search for files candidates for gzip
# according to extension and size
# this gzip text files in $1 dir only if newer

[[ $# -eq 1 ]] || exit 1 

find "${1}" -type f \
    -size +1500c -a \
    \( \
    -name "*\.html" -o \
    -name "*\.css" -o \
    -name "*\.js" -o \
    -name "*\.svg" -o \
    -name "*\.xml" -o \
    -name "*\.atom" -o \
    -name "*\.txt" \
    \) \
    | while read -r line; do
    echo "gzip $line"
    gzip -9 < "${line}" > "${line}.gz"
done

Use it this way :

$ gzip_site.sh /path/to/your/website

Reload httpd and enjoy

# rcctl restart httpd

FAQ

BUGS

Maybe, please tell me.