Deploying Nagare applications in a production environment¶
Serving a Nagare application in a development setup is easy. You just have to launch the following command:
nagare-admin serve [options] <app_name>
It starts a standalone threaded server which stores the sessions data in memory.
This standalone server is very convenient when developing Nagare applications, especially when
using the --reload and --debug options that respectively restart the server when source
files are changed and show the exceptions stacktraces as Web pages instead of in the console
shell.
However, this standalone mode is not suitable in a production environment, because it is not efficient nor fail-safe:
- Since it’s a threaded program, it does not fully utilize the CPU cores due to the GIL (see this discussion). To solve this problem, we need to use multiprocessing.
- Static files are served by the standalone server and are not cached. They would be better served by a faster dedicated Web server such as Apache, Nginx, Lighttpd …
- The server processes are not monitored, and they don’t cope with failures very well. For example, in a production setup, it’s important to detect zombie processes and restart them to prevent attacks or to limit the impact of unnoticed programmation errors in the application. Of course, you should also use logging to trace the origin of the problems in order to solve them.
So, in a production environment, the recommended way to serve Nagare applications is through FastCGI associated to a frontend Web server such as Apache, Nginx or Lighttpd. We are now going to describe how to do that.
Serving an application through FastCGI¶
In a FastCGI setup, a Web server such as Apache, Nginx or Lighttpd is needed. It will serve the static contents of the application, such as images, CSS and javascript files, and pass the other requests (i.e. those with a “dynamic” nature) to the Nagare application through the FastCGI protocol.
FastCGI is a variation of the CGI protocol, which defines an interface layer between a Web server and external applications (such as shell or python scripts). The main difference between FastCGI and CGI is that FastCGI doesn’t create a new worker process at the start of each request but use a pool of processes that are created when the FastCGI service starts, and uses sockets for exchanging informations (i.e. request and response data) between the Web server and the external applications.
This setup is particularly efficient when serving Nagare applications because the pool of processes fully utilize the CPU cores. Furthermore, Web servers are far more efficient than the standalone threaded server when serving the static contents of the application since they cache the data and they are mostly written in C.
The publisher configuration¶
In order to serve a Nagare application through FastCGI, we must first tell the nagare-admin serve utility
that we don’t want to use the standalone server but FastCGI instead. This is done by creating a publisher
configuration file (see The publisher configuration file) and passing it to the nagare-admin serve command
with the -c option.
Here is a typical publisher configuration file to serve an application through FastCGI:
[publisher]
type = fastcgi
host = 127.0.0.1
port = 9000
[sessions]
type = memcache
host = 127.0.0.1
port = 11211
We use the fastcgi backend for the publisher (with the default settings) and the memcache backend for
the sessions management.
Then, the FastCGI server can be launched by running this command:
nagare-admin serve -c publisher.conf <app_name>
By default, the FastCGI server forks 5 worker processes at startup to handle the incoming requests received on port 9000. However, the number of worker processes can grow or shrink depending on the workload.
Warning
When an application is served through FastCGI, you must use a memcache
backend for storing the sessions data. The default standalone session
backend will not work with FastCGI because the session data is stored
in memory, and since the processes spawned by the FastCGI server don’t
share memory (by definition), the sessions data stored in one process would
not be available in another process.
That’s why a kind of distributed “shared memory” session backend should be
used with the FastCGI publisher. So don’t forget to launch your memcached
server.
As the sessions are shared through the memcached server, the same FastCGI application can be launched on several machines thus easily creating a scalable applicative cluster.
Configuring the Web server¶
The next step is to configure the Web server so that it serves the static files and pass the other requests to the FastCGI processes we just launched. Of course, the procedure depend on the Web server used, so we are going to show you how to achieve that with Apache, Nginx and Lighttpd. However, it should be easy to use another Web server supporting FastCGI if you want to.
Apache¶
On Apache, the FastCGI support is provided by the mod_fastcgi module which is not installed by default.
You should download and install it by yourself, as described here.
Then, the mod_fastcgi module should be activated by including it in the main apache configuration file as shown
below:
LoadModule fastcgi_module /path/to/mod_fastcgi.so
The rewrite rules serving the static files of your application can be generated with this command:
nagare-admin create-rules --apache <application> > rewrite_rules.apache
You should obtain something like this:
RewriteEngine On
RewriteRule ^/static/nagare/(.*)$ /path/to/python/site-packages/nagare-0.5.0-py2.7.egg/static/$1 [L]
RewriteRule ^/static/myapp/(.*)$ /path/to/python/site-packages/myapp-0.0.1-py2.7.egg/static/$1 [L]
Then, you must create a VirtualHost for your application and include the rewrite rules file
into the configuration, as shown below:
# virtualhost configuration for http://www.myapp.com
<VirtualHost *:80>
ServerName www.myapp.com
FastCGIExternalServer /path/to/python/site-packages/fcgi -host 127.0.0.1:9000
Include /path/to/rewrite_rules.apache
RewriteRule ^/(.*)$ /fcgi/$1 [QSA,L]
</VirtualHost>
The FastCGIExternalServer directive instructs Apache to forward the requests directed to
/path/to/python/site-packages/fcgi to the FastCGI server listening at 127.0.0.1 on port
9000 which has been launched externally, and the last RewriteRule directive redirects
the requests not handled by the other rewrite rules to /path/to/python/site-packages/fcgi,
thus forwarding those requests to the FastCGI processes.
Finally, you should include the VirtualHost configuration file into the main
apache configuration file, or put it in the conf.d directory of your apache installation,
and restart Apache.
Nginx¶
Nginx has builtin support for FastCGI thanks to the Nginx’s HttpFastcgiModule, so no manual installation is necessary.
The rewrite rules serving the static files can be generated with this command:
nagare-admin create-rules --nginx <application> > rewrite_rules.nginx
You should obtain something like this:
location /static/nagare/ {
alias /path/to/python/site-packages/nagare-0.5.0-py2.7.egg/static/;
}
location /static/myapp/ {
alias /path/to/python/site-packages/myapp-0.0.1-py2.7.egg/static/;
}
Then, you must create a server configuration, as shown below:
server {
listen 80;
server_name www.myapp.com;
access_log /var/log/nginx/myapp.access.log;
error_log /var/log/nginx/myapp.error.log;
# serve the static files
include /path/to/rewrite_rules.nginx;
# serve the application
location / {
include /etc/nginx/fastcgi_params;
# Nagare applications need a properly set PATH_INFO variable
# either pass the request URI or use fastcgi_split_path_info to split the URI
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_pass 127.0.0.1:9000;
}
}
Note that, in addition to the default FastCGI parameters specified in /etc/nginx/fastcgi_params,
we must also set the PATH_INFO parameter which is required by Nagare.
Finally, you must copy the configuration file in /etc/nginx/conf.d/, or create a symbolic link
there pointing to your configuration file, and restart Nginx.
Lighttpd¶
Lighttpd has also builtin support for FastCGI thanks to the Lighttpd’s mod_fastcgi module.
The rewrite rules serving the static files can be generated with this command:
nagare-admin create-rules --lighttpd <application> > rewrite_rules.lighttpd
However, it’s easier the generate the rewrite rules in an include_shell directive.
An example of Lighttpd server configuration is show below:
server.modules = ( "mod_rewrite", "mod_fastcgi" )
server.errorlog = "/tmp/lighttpd.log"
server.indexfiles = ( "index.html" )
#debug.log-request-handling = "enable"
server.port = 8080
server.bind = "0.0.0.0"
#server.event-handler = "linux-sysepoll"
fastcgi.server = (
"/fcgi" =>
(
(
"host" => "127.0.0.1",
"port" => 9000,
"check-local" => "disable",
)
)
)
# Here, change the path to the ``nagare-admin`` command
include_shell "path/to/nagare-admin create-rules --lighttpd"
url.rewrite += (
"^(.*)" => "/fcgi/$1"
)
# ---------------------------------------------------------------------------------
mimetype.assign = (
".pdf" => "application/pdf",
".sig" => "application/pgp-signature",
".spl" => "application/futuresplash",
".class" => "application/octet-stream",
".ps" => "application/postscript",
".torrent" => "application/x-bittorrent",
".dvi" => "application/x-dvi",
".gz" => "application/x-gzip",
".pac" => "application/x-ns-proxy-autoconfig",
".swf" => "application/x-shockwave-flash",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".tar" => "application/x-tar",
".zip" => "application/zip",
".mp3" => "audio/mpeg",
".m3u" => "audio/x-mpegurl",
".wma" => "audio/x-ms-wma",
".wax" => "audio/x-ms-wax",
".ogg" => "application/ogg",
".wav" => "audio/x-wav",
".gif" => "image/gif",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".png" => "image/png",
".xbm" => "image/x-xbitmap",
".xpm" => "image/x-xpixmap",
".xwd" => "image/x-xwindowdump",
".css" => "text/css",
".html" => "text/html",
".htm" => "text/html",
".js" => "text/javascript",
".asc" => "text/plain",
".c" => "text/plain",
".conf" => "text/plain",
".text" => "text/plain",
".txt" => "text/plain",
".dtd" => "text/xml",
".xml" => "text/xml",
".mpeg" => "video/mpeg",
".mpg" => "video/mpeg",
".mov" => "video/quicktime",
".qt" => "video/quicktime",
".avi" => "video/x-msvideo",
".asf" => "video/x-ms-asf",
".asx" => "video/x-ms-asf",
".wmv" => "video/x-ms-wmv",
".bz2" => "application/x-bzip",
".tbz" => "application/x-bzip-compressed-tar",
".tar.bz2" => "application/x-bzip-compressed-tar"
)
Then, place this config file in /etc/lighttpd/lighttpd.conf and restart Lighttpd.
Handling the FastCGI processes¶
Since the FastCGI processes are launched externally from the Web server, you can use any monitoring tool to handle the processes, such as Supervisor. Here is a typical configuration file that starts a Nagare application as a daemon with Supervisor:
[unix_http_server]
file=/path/to/supervisord/supervisor.sock
[supervisord]
logfile=/path/to/supervisord/supervisord.log
pidfile=/path/to/supervisord/supervisord.pid
directory=/path/to/supervisord
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///path/to/supervisord/supervisor.sock
[program:myapp]
command=/path/to/nagare-admin serve /path/to/application/conf/myapp.conf -c /path/to/application/conf/fastcgi.conf
process_name=myapp
autostart=true
autorestart=true
stdout_logfile=/path/to/application/logs/myapp.log
redirect_stderr=true
Then, you can start the application by running this command:
/path/to/supervisord -c /path/to/supervisord.conf
The supervisord process daemonizes itself and starts the FastCGI processes through
the nagare-admin serve command. Furthermore, they are restarted automatically when something goes
wrong thanks to the autorestart option.