# DeviceAtlas C API and NGINX Module Installation # Provides detailed instructions to build, install and use the DeviceAtlas NGINX module. 1. Dependencies 2. DeviceAtlas API and NGINX packages - Building DeviceAtlas library - Option 1 - static module - Option 2 - dynamic (.so) module - Option 3 - dynamic module for NGINX Plus 3. NGINX setup with DeviceAtlas 4. DeviceAtlas data file reload 5. Example configurations 6. Example application (Python FCGI) 7. Debugging 8. Common errors ### Dependencies ### Only the DeviceAtlas C/NGINX API package and NGINX source code (1.10.1 or later) are required to build the module (see below for details). The DeviceAtlas C API requires the PCRE regex library to be available. ### DeviceAtlas API and NGINX packages ### ** Get the DeviceAtlas C/NGINX API package ** - Minimum version: 2.1.1 The API package can be downloaded from the [deviceatlas.com](https://deviceatlas.com/resources/download-enterprise-api) website. A login and appropriate permissions are required to download. Please contact sales@deviceatlas.com if you are not sure about access. Once the package is downloaded, extract the files to a local folder. ```sh tar -zxvf deviceatlas_c-nginx_{version}.tar.gz ``` ** Get the NGINX source code ** - Minimum NGINX version: 1.10.1 or later If not available in the local environment, download the source code from the [nginx.org](http://nginx.org/en/download.html) website and extract to a local folder. ```sh wget "http://nginx.org/download/nginx-1.10.2.tar.gz" tar -zxvf nginx-1.10.2.tar.gz ``` ### Building DeviceAtlas API library ### The DeviceAtlas NGINX module requires that the DeviceAtlas API library (libda.so) is built and installed in the OS. To build the library and install libda.so the following commands can be used: ```sh cd deviceatlas_c-nginx_{version}/Src cmake . make sudo make install sudo ldconfig ``` Please see the main README.DeviceApi.txt for further details. #### Using API with a custom library location #### If `make install` cannot be used then the compiled DeviceAtlas library can be placed in a custom location. The following NGINX configure flag must be used to reference the DeviceAtlas library: ``` --with-ld-opt='-Wl,-rpath,<DeviceAtlas C API folder>/Src' ``` ### Building NGINX module ### The DeviceAtlas module can be built as either a `static` or `dynamic` module. The following sections describes how to build and run the module when built in either of these two forms. The dynamic module provides more flexibility and means the module can be used with pre-built NGINX binaries provided by the System and also with NGINX Plus. #### Option 1 - static module #### The static option compiles the DeviceAtlas module into an NGINX binary along with the NGINX source code. ** Configure NGINX source and build it with DeviceAtlas ** ```sh cd nginx-{version} ./configure --add-module=<DeviceAtlas C API folder>/Nginx-Mod make sudo make install ``` ** Starting NGINX ** NGINX can be launched by using the binary from the local NGINX installation: ```sh /usr/local/nginx/sbin/nginx ``` With a `nginx.conf` file located at `/usr/local/nginx/conf/nginx.conf`. The file locations may vary based on the installation process. #### Option 2 - dynamic (.so) module #### The second option is to build as a dynamic module. This creates a .so file separate from the main NGINX build. **Important 1** - The version of the downloaded NGINX source code must match the installed target NGINX binary for the built dynamic module to be compatible. To confirm the version use `nginx -v`. **Important 2** - The same flags as the target NGINX binary must be used when compiling the module to ensure compatibility. ** Configure NGINX source and build the DeviceAtlas module ** As mentioned above it is essential to compile with the same flags as the existing NGINX binary. The flags used to compile the NGINX binary can be found as follows: ```sh nginx -V ``` Below is the **sample** output from the above command and is only for illustrative purposes. It should not be used as is. ```sh configure arguments: \ --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --modules-path=/usr/lib/nginx/modules \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --http-client-body-temp-path=/var/cache/nginx/client_temp \ --http-proxy-temp-path=/var/cache/nginx/proxy_temp \ --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \ --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \ --http-scgi-temp-path=/var/cache/nginx/scgi_temp \ --user=nginx \ --group=nginx \ --with-http_ssl_module \ --with-http_realip_module \ --with-http_addition_module \ --with-http_sub_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_mp4_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_random_index_module \ --with-http_secure_link_module \ --with-http_stub_status_module \ --with-http_auth_request_module \ --with-http_xslt_module=dynamic \ --with-http_image_filter_module=dynamic \ --with-http_geoip_module=dynamic \ --with-http_perl_module=dynamic \ --add-dynamic-module=debian/extra/njs-1c50334fbea6/nginx \ --with-threads \ --with-stream \ --with-stream_ssl_module \ --with-http_slice_module \ --with-mail \ --with-mail_ssl_module \ --with-file-aio \ --with-ipv6 \ --with-http_v2_module \ --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' \ --with-ld-opt='-Wl,-z,relro -Wl,--as-needed' ``` To include the DeviceAtlas module it must be added to the list of flags. There is one line to be added: ```sh --add-dynamic-module=<DeviceAtlas C API folder>/Nginx-Mod ``` The above output is modified to include the DeviceAtlas module as follows: (This is an example, the flags used for your NGINX may be different!) ```sh ./configure \ --add-dynamic-module=<DeviceAtlas C API folder>/Nginx-Mod \ --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --modules-path=/usr/lib/nginx/modules \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --http-client-body-temp-path=/var/cache/nginx/client_temp \ --http-proxy-temp-path=/var/cache/nginx/proxy_temp \ --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \ --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \ --http-scgi-temp-path=/var/cache/nginx/scgi_temp \ --user=nginx \ --group=nginx \ --with-http_ssl_module \ --with-http_realip_module \ --with-http_addition_module \ --with-http_sub_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_mp4_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_random_index_module \ --with-http_secure_link_module \ --with-http_stub_status_module \ --with-http_auth_request_module \ --with-http_xslt_module=dynamic \ --with-http_image_filter_module=dynamic \ --with-http_geoip_module=dynamic \ --with-http_perl_module=dynamic \ --add-dynamic-module=debian/extra/njs-1c50334fbea6/nginx \ --with-threads \ --with-stream \ --with-stream_ssl_module \ --with-http_slice_module \ --with-mail \ --with-mail_ssl_module \ --with-file-aio \ --with-ipv6 \ --with-http_v2_module \ --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' \ --with-ld-opt='-Wl,-z,relro -Wl,--as-needed' ``` ** Code compilation ** After running the `./configure` command the next step is compilation to get the final dynamic module (.so) file. ```sh make ``` Once built, the `ngx_http_deviceatlas_module.so` dynamic module file should be found in the `nginx-{version}/objs/` folder. Please be aware that it may be necessary to install additional libraries on your system (e.g. openssl, libxml2, GD library, ...) should other NGINX modules require them. #### Option 3 - dynamic module for NGINX Plus #### Building a module for NGINX Plus is very similar to building the module for NGINX binary. The process is simpler as only a handful of configuration flags are required. Again, it is important to note that the downloaded source code version must be identical to the version of the installed NGINX Plus binaries. To confirm the version use `nginx -v`. ** Configure NGINX source and build the DeviceAtlas module ** To make the module compatible it is necessary to compile it with the `--with-compat` flag and with the same flags which were used for to build the existing NGINX Plus binary. This can be accomplished by using NGINX executable like this: ```sh nginx -V ``` The important options are `--with-cc-opt` and `--with-ld-opt` which need to be copied over (this is only for illustrative purposes, not to be used as is), along with paths to the DeviceAtlas C API source files. ```sh ./configure \ --with-compat \ --add-dynamic-module=<DeviceAtlas C API folder>/Nginx-Mod \ --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' \ --with-ld-opt='-Wl,-z,relro -Wl,--as-needed' ``` ** Code compilation ** As before, the next step is compilation to get the final dynamic module (.so) file. ```sh make ``` Once built, the `ngx_http_deviceatlas_module.so` dynamic module file should be found in the `nginx-{version}/objs/` folder. ** Modules repository ** The module file should be placed in the general NGINX Plus modules folder: ```sh cp objs/ngx_http_deviceatlas_module.so /usr/lib/nginx/modules ``` ### NGINX setup with DeviceAtlas ### To use the module, you will need to download the required data file, save it locally and set the `devatlas_db` path in `nginx.conf`. To get the data file for device detection, please see the [DeviceApi](README.DeviceApi.html) readme file. ** A snippet from nginx.conf ** ```sh # Uncomment next line and fix the path if using dynamic module. #load_module /path/to/ngx_http_deviceatlas_module.so; http { variables_hash_max_size 2048; ## DeviceAtlas Detection devatlas_db /path/to/DeviceAtlas.json; ## DeviceAtlas variables prefix (optional) # All DeviceAtlas properties are prefixed with da_ and are available # as NGINX's environment variables (all dots are replaced with underscores). # Uncomment `devatlas_property_prefix` line to set a custom prefix for # the DeviceAtlas variables (default is da_). #devatlas_property_prefix http_; ## Property filter (optional) # By default the DeviceAtlas module registers all available properties # from the JSON file. If at least one `devatlas_property` definition is # found the module will register only those properties which are defined. # For a list of all properties see https://deviceatlas.com/properties devatlas_property primaryHardwareType; devatlas_property osName; devatlas_property osVersion; devatlas_property browserName; devatlas_property browserVersion; ## Client-side Component (optional) # To use the Client-side Component with the NGINX module, the JavaScript # file located in ExtraTools/ClientSide/ must be included on every webpage. # The JS library will create a cookie with the client properties. The # NGINX module reads this cookie and merges the properties with the server # side detected properties. devatlas_properties_cookie DAPROPS; server { listen 8080; server_name localhost; location / { fastcgi_param REQUEST_METHOD $request_method; fastcgi_param QUERY_STRING $query_string; # DeviceAtlas properties # nginx environment variables are formed as follow # <prefix><property name> e.g. da_model fastcgi_param DA_PRIMARY_HARDWARE_TYPE $da_primaryHardwareType; fastcgi_param DA_OS_NAME $da_osName; fastcgi_param DA_OS_VERSION $da_osVersion; fastcgi_param DA_BROWSER_NAME $da_browserName; fastcgi_param DA_BROWSER_VERSION $da_browserVersion; # More NGINX properties here } } } ``` ### DeviceAtlas data file reload ### The DeviceAtlas data file can be reloaded in place without restarting NGINX. Replace the on-disk file with a new data file downloaded from deviceatlas.com and trigger an NGINX reload. NGINX will continue to serve requests while the new data file is being loaded. An NGINX reload can be triggered by calling `nginx reload` or `service nginx reload`. The reload will ignore corrupt or missing data files and continue using the previous in-memory version. The NGINX error log will contain an error message if this occurs. ### Example configurations ### There are multiple NGINX configuration files and snippets available at `Nginx-Mod/Example` folder for common use-cases. - Hello World - Non-human traffic detection - Non-human traffic detection (with LUA) - Image optimisation - Mobile redirection - Language redirection - Passing variables downstream as headers #### Common settings (for all examples) #### All examples below requires following configuration. ** In main block (i.e. outside of `HTTP` and `server` blocks): ** ``` # Load DeviceAtlas module (Dynamic module option only) load_module /path/to/ngx_http_deviceatlas_module.so; ``` ** In HTTP block: ** ``` variables_hash_max_size 2048; devatlas_db /path/to/DeviceAtlas.json; ``` #### EXAMPLE 1 - Hello World #### This is a very simple configuration to ensure that DeviceAtlas is working (like Hello World, for the module). The path `/DeviceAtlasTest` will return a plain text response containing the device type e.g. "Mobile Phone" or "Tablet". ** In HTTP block: ** ``` # Assign DeviceAtlas properties as maps. # (It's recommended to always use default values as follows.) map $da_primaryHardwareType $primaryHardwareType { default $da_primaryHardwareType; "" "No primary hardware type available"; } ``` ** In server block: ** ``` location /DeviceAtlasTest { add_header Content-Type text/plain; return 200 $primaryHardwareType; } ``` #### EXAMPLE 2 - Non-human traffic detection #### This configuration shows how you might treat traffic for known bots. In this case they are redirected to a separate page but alternate content could be served also. ** In server block: ** ``` location / { default_type 'text/plain'; content_by_lua_block { /* General property to detect all types of bots traffic */ local isNonHumanTraffic = ( ngx.var.da_isRobot == "1" ) if isNonHumanTraffic then ngx.say("You are a bot !!") else /* Creation of an environement variable to store the hardware type */ local primaryHardwareType = 'generic' if ngx.var.da_primaryHardwareType then primaryHardwareType = ngx.var.da_primaryHardwareType end set_by_lua $primary_hardware_type primaryHardwareType end } } ``` #### EXAMPLE 3 - Non-human traffic detection (with LUA) #### This configuration shows how you might treat traffic for known bots. In this case they are redirected to a separate page but alternate content could be served also. ** In HTTP block: ** ``` # Assign DeviceAtlas properties as maps. # (It's recommended to always use default values as follows.) map $da_isRobot $robot { true 1; false 0; default 0; } ``` ** In server block: ** ``` # Redirect robots to scraping policy page if ($robot) { rewrite ^ http://www.example.com/scrapingpolicy.html; } ``` #### EXAMPLE 4 - Image optimisation - part #1 #### This example demonstrates how you might resize images to maximum display size for a given device. Desktop devices will see no change, mobile devices should be served images appropriate to their resolutions. Resized images are cached to ensure minimal load on server (one resize operation per target resolution). This is part #1, see part #2 for an actual resizer listening at port 9000. ** In HTTP block: ** ``` # Assign DeviceAtlas properties as maps. # (It's recommended to always use default values as follows.) map $da_usableDisplayWidth $displayWidth { default $da_usableDisplayWidth; "" "-"; } map $da_usableDisplayHeight $displayHeight { default $da_usableDisplayHeight; "" "-"; } ``` ** In server block: ** ``` location ~* "^/images/(.*)" { set $image_path $1; proxy_pass http://127.0.0.1:9000/image_resizer/$image_path?width=$displayWidth&height=$displayHeight; proxy_cache resized; proxy_cache_valid 180m; } ``` #### Image optimisation - part #2 #### ** In HTTP block: ** ``` limit_req_zone "1" zone=2persec:32k rate=2r/s; ``` ** In server block: ** ``` listen 9000; allow 127.0.0.1; deny all; limit_req zone=2persec burst=10; location /image_resizer { alias /usr/share/nginx/html/path/to/images; image_filter resize $arg_width $arg_height; } ``` #### EXAMPLE 5 - Mobile redirection #### This example demonstrates how you might rewrite URLS to serve mobile-specific content (e.g. AMP pages) to certain device classes. ** In HTTP block: ** ``` # Assign DeviceAtlas properties as maps. # (It's recommended to always use default values as follows.) map $da_mobileDevice $mobile { true 1; false 0; default 0; } ``` ** In server block: ** ``` # Send mobile device to AMP version of content if ($mobile) { rewrite ^(.*)$ /AMP/$1 break; } ``` #### EXAMPLE 6 - Language redirection #### This example demonstrates how you might rewrite URLS based on a client's detected language. ** In HTTP block: ** ``` # Assign DeviceAtlas properties as maps. # (It's recommended to always use default values as follows.) map $da_language $language { default $da_language; "" "en"; } ``` ** In server block: ** ``` location / { rewrite ^(.*)$ /$language/$1; } ``` #### EXAMPLE 7 - Passing variables downstream as headers #### This example demonstrates how you might pass variables downstream via request headers. ** In server block (LB): ** ``` location /DeviceAtlasTest { proxy_set_header X-DeviceAtlas-isMobilePhone $da_isMobilePhone; proxy_pass http://localhost:8080; } ``` ** In HTTP block (downstream): ** In case of underscore headers, set by LB, an extra directive is needed to be set on the downstream servers. e.g. proxy_set_header DA_IS_MOBILE_PHONE $da_isMobilePhone; This is not needed if the header names are set with dashes as `X-DeviceAtlas-<variable>` (as is in the example above). ``` underscores_in_headers on; ``` ### Example application ### There is a sample python application in the `Nginx-Mod/Example` folder. It consists of a small python script `app.py` which prints the list of all available DeviceAtlas properties, CSS file and the Client-side Component. To make the static files available for NGINX copy the static files folder to the default NGINX `html` location. ```sh cp -r <DeviceAtlas C API folder>/Nginx-Mod/Example/static /usr/local/nginx/html/static ``` #### Data file #### To use the module, you will need to download a data file, save it locally and set the path from **deviceatlas.nginx.conf** using `devatlas_db` directive. To get a data file for Device Detection, please see the [DeviceApi](README.DeviceApi.html) readme file. #### Run NGINX server #### ```sh /usr/local/nginx/sbin/nginx -c <DeviceAtlas C API folder>/Nginx-Mod/Example/deviceatlas.nginx.conf ``` NGINX is now up and running on port 8080. #### Run example app (python3) #### 1. Open up new terminal There is a very simple example app bundled with the source code which demonstrates how properties are passed back to your application. It is located in the nginx/app.py and you will need a python Flup module to run it. 2. Install Flup (if not installed already) ```sh easy_install3 flup ``` 3. Run example app directly with python: ```sh python3 <DeviceAtlas C API folder>/Nginx-Mod/Example/app/app.py ``` Then, when you access the URL http://localhost:8080/app/, it will display all values from the device property list specified in the **deviceatlas.nginx.conf**. If you change the user-agent in your browser settings, you can then see the device properties for a device being reported. ### Optional ### ** Client-side Component ** To use the Client-side Component with the NGINX module, a Javascript file located in ExtraTools/ClientSide/ must be included on every webpage. The component will create a cookie with the client properties. The NGINX module automatically reads this cookie and merges the properties with the server side detected properties. For more information please see the [Client-side Component](README.ClientSide.html) readme file. ### Debugging ### ** Is DeviceAtlas module running? ** The simplest way to confirm if the DeviceAtlas module is loaded successfully is by looking in NGINX's error.log (typically in `/var/log/nginx/error.log` or `/usr/local/nginx/logs/error/log`). It will have "[notice]" lines similar to the following in it: ``` [notice] DeviceAtlas cookie name 'DAPROPS' in /etc/nginx/nginx.conf:57 [notice] DeviceAtlas, environment variable 'da_primaryHardwareType' added (property 'primaryHardwareType') in /etc/nginx/nginx.conf:57 [notice] DeviceAtlas, environment variable 'da_osName' added (property 'osName') in /etc/nginx/nginx.conf:57 [notice] DeviceAtlas, environment variable 'da_osVersion' added (property 'osVersion') in /etc/nginx/nginx.conf:57 ``` ** Checking DeviceAtlas values ** The easiest way to debug issues is to add the DeviceAtlas properties to the NGINX log file so that you can see their value for each request. ``` log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' '"$da_mobileDevice" "$da_vendor $da_model"'; ``` ### Common errors ### ** Version compatibility ** To prevent the error below ensure that the version of the source NGINX matches the existing built NGINX version (e.g. provided by the OS). ```sh nginx: [emerg] module "/usr/lib/nginx/modules/ngx_http_deviceatlas_module.so" version 1011008 instead of 1011005 ``` ** Binary compatibility ** To prevent the error below make sure the same configuration flags used by the OS's NGINX binary version were used while building the dynamic module. ```sh nginx: [emerg] module "/usr/lib/nginx/modules/ngx_http_deviceatlas_module.so" is not binary compatible in /etc/nginx/nginx.conf ``` - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - _ Copyright (c) DeviceAtlas Limited 2021. All Rights Reserved. _ https://deviceatlas.com <!-- HTML+JS for document formatting when opened in browser --> <div class="btn-group" id="main-menu" style="float:right"><a class="btn dropdown-toggle" data-toggle="dropdown" href="#">Menu<span class="caret"></span></a><ul class="dropdown-menu"><li><a href="README.html">Main</a></li><li class="divider"></li><li><a href="README.DeviceApi.html">Device Detection API</a></li><li class="divider"></li><li class="disabled"><a href="README.Nginx.html">C API and NGINX Module Installation</a></li><li><a href="README.ClientSide.html">Client-side Component</a></li><li class="divider"></li><li><a href="ApiDocs/index.html">DeviceAtlas ApiDocs</a></li></ul></div>