Uwsgi

Changelog [20131011]

License change
**************

This version of uWSGI is the first of the 1.9 tree using GPL2 + linking exception instead of plain GPL2.

This new license should avoid any problems when using uWSGI as a shared library (or when linking it with non-GPL2 compatible libraries)

Remember: if you need to make closed-source modifications to uWSGI you can buy a commercial license.

Bugfixes
********

- fixed uwsgi native protocol support on big endian machines
- fixed jvm build system for arm (Jorge Gallegos)
- fixed a memleak spotted by cppcheck in zlib management
- chdir() at every emperor glob iteration
- correctly honour --force-cwd
- fixed ia64/Linux compilation (Jonas Smedegaard/Riccardo Magliocchetti)
- fixed ruby rvm paths parsing order
- added waitpid() after daemon's SIGTERM (Łukasz Mierzwa)
- fixed pid numbering after --idle (Łukasz Mierzwa)
- fixed/improved cheaper memory limits (Łukasz Mierzwa)
- correctly close inherited sockets in gateways
- fix checks for MAP_FAILED in mmap() (instead of NULL)
- fixed FastCGI non-blocking body read() (patch by Arkaitz Jimenez)
- fixed attach.py script
- avoid crashing on non-conformant PSGI response headers
- run the python autoreloader even in non-apps mode when non-lazy

New Features
************

Minimal build profiles
^^^^^^^^^^^^^^^^^^^^^^

Albeit the memory usage of the uWSGI core is generally between 1.8 and 2.5 megs, there are use cases in which you want an even minimal
core and set of embedded plugins.

Examples are users not making use of uWSGI specific features, or cases in which the libraries used by uWSGI nameclash with others (like openssl or zeromq).

A bunch of 'minimal' build profiles have been added:

* pyonly (build a minimal CPython WSGI server)
* pypyonly (build a minimal PyPy WSGI server)
* plonly (build a minimal PSGI server)
* rbonly (build a minimal Rack server)

the only supported configuration format is .ini and internal routing and legion subsystem are not builtin.

For example if you want to install a minimal uWSGI binary via pip:

.. code-block:: sh

UWSGI_PROFILE=pyonly pip install uwsgi

IMPORTANT: minimal build profiles do not improve performance, for the way uWSGI is designed, unused features do not waste CPU. Minimal build profiles impact on final binary size only

Auto-fix modifier1
^^^^^^^^^^^^^^^^^^

Setting the modifier1 for non-python plugin is pretty annoying (read: you always forget about it).

Now if the modifier1 of the request is zero, but the python plugin is not loaded (or there are no python apps loaded) the first configured app
will be set instead (unless you disable with feature with --no-default-app).

This means you can now run:

.. code-block:: sh

uwsgi --http-socket :9090 --psgi myapp.pl

instead of

.. code-block:: sh

uwsgi --http-socket :9090 --http-socket-modifier1 5 --psgi myapp.pl

obviously try to always set the modifier1, this is only a handy hack

Perl auto reloader
^^^^^^^^^^^^^^^^^^

The --perl-auto-reload option allows the psgi plugin to check for changed modules after every request. It takes the frequency (in seconds) of the scan.

The scan happens after a request has been served. It is suboptimal, but it is the safest choice too.

The "raw" mode (preview technology, only for CPython)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

While working on a new server-side project in Unbit we had the need to expose our web application using a very specific protocol (none of the ones supported by uWSGI).

Our first way was adding the new protocol as a plugin, but soon we realize that is was too-specific. So we decided to introduce the RAW mode.

Raw mode allows you to directly parse the request in your application callable. Instead of getting a list of CGI vars/headers in your callable
you only get the file descriptor soon after accept().

You can then read()/write() to that file descriptor in full freedom.

.. code-block:: python

import os
def application(fd):
os.write(fd, "Hello World")

.. code-block:: sh

uwsgi --raw-socket :7070 --python-raw yourapp.py

Raw mode disables request logging. We currently support it only for CPython, if we get reports (or interest) about it for the other languages we will add
support for sure.

IMPORTANT: raw mode is not a standard, so do not expect any middleware or common usage patterns will apply. Use it as a low-level socket wrapper.



Optional NON-standard support for CPython buffer protocol for WSGI responses
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Authors: yihuang with help of INADA Naoki (methane)

The WSGI (PEP333/3333) is pretty clear about the type of valid objects for responses: str for python2, bytes for python3

uWSGI (heavily using mod_wsgi as a reference) always enforce such behaviour, so "exotic" patterns like returning bytearray
where not supported. Such uses are somewhat involuntary supported on pure-python application servers, just because they simply call write() over them or because they cast them to string
before returning (very inefficient)

The patch proposed by yihuang suggests the use of the low-level buffer protocol exposed by the CPython C api. Strings (in python2) and bytes (in python3) support the buffer protocol, so its use is transparent
and backward compatibility is granted too. (for the CPython C api experts: yes we support both old and new buffer protocol)

This is a NON-standard behaviour you have to voluntary enable with --wsgi-accept-buffer.

Use with care as it could mask errors and/or wrong behaviours.

Note: if you tried 1.9.18-dev you may note this option was enabled by default. It was an error. Thanks to Graham Dumpleton (mod_wsgi author) for pointing it out.

Emperor and config improvements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Credits: Matthijs Kooijman

The config system has been improved to be even more consistent in respect to strict mode (remainder: with --strict you basically check your config files for unknown options
avoiding headaches caused by typos).

New magic vars have been added exposing the name of the original config file (this simplify templating when in Emperor mode), check them at https://github.com/unbit/uwsgi-docs/blob/master/Configuration.rst#magic-variables

The Emperor got support for Linux capabilities using the --emperor-cap option. The option takes the list of capability you want to maintain
for your vassals when they start as root:

.. code-block:: ini

[uwsgi]
emperor = /etc/uwsgi/vassals
emperor-cap = setuid,net_bind_service

with this setup your vassal will be only able to drop privileges and bind to ports < 1024

Its best friend is the CLONE_NEWUSER flag of linux namespaces that is now fully supported on uWSGI:

.. code-block:: ini

[uwsgi]
emperor = /etc/uwsgi/vassals
emperor-use-clone = user
emperor-cap = setuid,net_bind_service

this will create a new root user for the vassal with fewer privileges (CLONE_NEWUSER is pretty hard to understand, but the best thing
to catch it is seeing it as a new root user with dedicated capabilities)

Build system improvements
^^^^^^^^^^^^^^^^^^^^^^^^^

The build system has been improved to link custom sources on the fly. This works great for low-level hooks:

.. code-block:: c

// embed_me.c
#include <stdio.h>

void hello_i_am_foobar() {
printf("I Am foobar");
}

Now we can link this file to the main uWSGI binary in one shot:


.. code-block:: sh

UWSGI_ADDITIONAL_SOURCES=embed_me.c make

and you will automatically get access for your hooks:

.. code-block:: sh

uwsgi --http-socket :9090 --call-asap hello_i_am_foobar

Finally, Riccardo Magliocchetti rewrote the build script to use optparse instead of raw/old-fashioned sys.argv parsing


Pluginized the 'schemes' management
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

schemes are the prefix part of uWSGI uri's. When you do

.. code-block:: sh

uwsgi --ini http://foobar.local:9090/test.ini

the http:// is the scheme, signalling uWSGI it has to download the config file via http.

Til now those 'schemes' were hardcoded. Now they are exposed as plugins, so you can add more of them (or override the default one).

The new system has been applied to the PSGI plugin too (sorry we are sure only perl developers will understand that kind of poetry :P) so you can do things like:

.. code-block:: sh

uwsgi --http-socket :1717 --psgi http://yourapps.local/dancer.pl

or

.. code-block:: sh

./uwsgi --binary-append-data yourapp.pl > blob001
cat blob001 >> ./uwsgi
./uwsgi --http-socket :1717 --psgi data://0

mountpoints checks
^^^^^^^^^^^^^^^^^^

It could be hard to understand why an application server should check for mountpoints.

In the same way understanding how writing filesystem in userspace was silly few years ago.

So, check the article about managing Fuse filesystem with uWSGI: https://uwsgi-docs.readthedocs.io/en/latest/tutorials/ReliableFuse.html

Preliminary libffi plugin
^^^^^^^^^^^^^^^^^^^^^^^^^

As embedding c libraries for exposing hooks is becoming more common, we have started working on libffi integration, allowing
safe (and sane) argument passing to hooks. More to came soon.

Official support for kFreeBSD
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Debian/kFreeBSD is officially supported.

You can even use FreeBSD jails too !!!

:doc:`FreeBSDJails`

Availability
************

uWSGI 1.9.18 has been released on October 11th 2013 and can be downloaded from:

https://projects.unbit.it/downloads/uwsgi-1.9.18.tar.gz

Changelog [20130917]


Bugfixes
********

- the 'pty' client is now blocking (safer approach)
- removed strtok() usage (substituted by a new uwsgi api function on top of strtok_r() )
- fixed --pty-exec (Credits: C Anthony Risinger)
- listen_queue/somaxconn linux check is now done even for UNIX sockets



New features
************

The Master FIFO
^^^^^^^^^^^^^^^

This is a new management way in addition to UNIX signals. As we have no more free signals to use (and generally dealing with signals and pidfiles is not very funny), all of the new management features of uWSGI will be based on the master fifo.

Docs are already available: :doc:`MasterFIFO`


The asap hook
^^^^^^^^^^^^^

Credits: Matthijs Kooijman

a new hook, named 'asap' has been added. It will be run soon after the options are parsed.

Check: :doc:`Hooks`

The TCC (libtcc) plugin
^^^^^^^^^^^^^^^^^^^^^^^

TCC is an embeddable c compilers. It includes a shared library (libtcc) you can use to compile strings of c code on the fly.

The libtcc uWSGI plugins allows compiling strings of c to process symbols. CUrrently the "tcc" hook engine has been implemented:

.. code-block:: ini

[uwsgi]
hook-asap = tcc:mkdir("/var/run/sockets");printf("directory created\n");
hook-as-user = tcc:printf("i am process with pid %d\n", getpid());
hook-post-app = tcc:if (getenv("DESTROY_THE_WORLD")) exit(1);
http-socket = /var/run/sockets/foobar.sock



The forkptyrouter gateway
^^^^^^^^^^^^^^^^^^^^^^^^^

While work on Linux containers/namespaces continues to improve we have added this special router/gateway allowing dynamic allocation of pseodoterminals
in uWSGI instances. To access the sockets created by the forkptyrouter you can use the --pty-connect option exposed by the 'pty' plugin.

Documention is being worked on.

added a new magic var for ANSI escaping
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The %[ magic var has been added, it allows you to define ANSI sequences in your logs.

If you like coloured logs:

.. code-block:: ini

log-encoder = format %[[33m${msgnl}%[[0m

Routable log encoders
^^^^^^^^^^^^^^^^^^^^^

You can now attach log encoders to specific log routes:

.. code-block:: ini

[uwsgi]
logger = stderr file:/dev/tty
log-route = stderr ubuntu
log-route = stderr clock
print = %[[34mHELLO%[[0m
; add an encoder to the 'stderr' logger
log-encoder = format:stderr %[[33m${msgnl}%[[0m
http-socket = :9090

--vassals-include
^^^^^^^^^^^^^^^^^

Credits: Matthijs Kooijman

This is like --vassal-inherit but the parsing will be "immediate" (so you can use placeholders)

The Emperor heartbeat system is now mercyless...
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The old approach for the heartbeat Emperor subsystem was asking for "gentle" reload to bad vassals.

Now vassals not sending heartbeat (after being registered with the heartbeat subsystem) are killed with -9

The result of this patch will be more robust bad vassals management

logpipe
^^^^^^^

Author: INADA Naoki

You can now send loglines to the stdin of an external command:

.. code-block:: ini

req-logger = pipe:/usr/local/bin/mylogger

added "fd" logger to "logfile" plugin
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

you can directly send logs to a file descriptors:

.. code-block:: ini

req-logger = fd:17


Availability
************

uWSGI 1.9.17 has been released on Semptember 22th 2013

You can download it from:

https://projects.unbit.it/downloads/uwsgi-1.9.17.tar.gz

Changelog [20130914]


Important change in the gevent plugin shutdown/reload procedure !!!
*******************************************************************

The shutdown/reload phase when in gevent mode has been changed to better integrate
with multithreaded (and multigreenlet) environments (most notably the newrelic agent).

Instead of "joining" the gevent hub, a new "dummy" greenlet is spawned and "joined".

During shutdown only the greenlets spawned by uWSGI are taken in account, and after all of them are destroyed
the process will exit. This is different from the old approach where the process wait for ALL the currently available greenlets
(and monkeypatched threads).

If you prefer the old behaviout just specify the option --gevent-wait-for-hub


Bugfixes/Improvements
*********************

- fixed CPython reference counting bug in rpc and signal handlers
- improved smart-attach-daemon for slow processes
- follow Rack specifications for QUERY_STRING,SCRIPT_NAME,SERVER_NAME and SERVER_PORT
- report missing internal routing support (it is only a warning when libpcre is missing)
- better ipcsem support during shutdown and zerg mode (added --persistent-ipcsem as special case)
- fixed fastcgi bug exposed by apache mod_fastcgi
- do not call pre-jail hook on reload
- force linking with -lrt on solaris
- report thunder lock status
- allow custom priority in rsyslog plugin

New features
************

FreeBSD jails native support
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

uWSGI got nativr FreeBSD jails support. Official documentation is here :doc:`FreeBSDJails`

The Rados plugin
^^^^^^^^^^^^^^^^

Author: Javier Guerra

Based on the :doc:`GlusterFS` plugin, a new one allowing access to Rados object storage is available:

:doc:`Rados`

The TunTap router
^^^^^^^^^^^^^^^^^

This new gateway is the result of tons of headaches while trying to build better (read: solid) infrastructures with Linux namespaces.

While dealing with uts, ipc, pid and filesystem namespaces is pretty handy, managing networking is a real pain.

We introduced lot of workaroud in uWSGI 1.9.15 (especially to simplify the veth management) but finally we realized
that those systems do not scale in terms of management.

The TunTap router tries to solve the issue moving the networking part of jailed vassals in user space.

Basically each vassal create one or more tuntap devices. This devices are connected (via a unix socket) to the "tuntap router"
allowing access from the vassal to the external network.

That means a single network interface in the main namespace and one for each vassal.

The performance are already quite good (we are only losing about 10% in respect of kernel-level routing) but can be optimized.

In addition to this the tuntap router has a simple userspace firewall you can use to manage complex routing rules.

Documentation is still in progress, but you can configure a tuntap router following the big comment on top of this file:

https://github.com/unbit/uwsgi/blob/master/plugins/tuntap/tuntap.c

while you can connect to it with ``--tuntap-device <dev> <socket>`` where <dev> is the tuntap device to create in the vassal/client and <socket> is the unix address
of the tuntap router

An Example Emperor

.. code-block:: ini

[uwsgi]
tuntap-router = emperor0 /tmp/tuntap.socket
exec-as-root = ifconfig emperor0 192.168.0.1 netmask 255.255.255.0 up
exec-as-root = iptables -t nat -F
exec-as-root = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
exec-as-root = echo 1 >/proc/sys/net/ipv4/ip_forward
emperor-use-clone = ipc,uts,fs,pid,net
emperor = /etc/vassals

and one of its vassals:

.. code-block:: ini

[uwsgi]
tuntap-device = uwsgi0 /tmp/tuntap.socket
exec-as-root = ifconfig lo up
exec-as-root = ifconfig uwsgi0 192.168.0.2 netmask 255.255.255.0 up
exec-as-root = route add default gw 192.168.0.1
exec-as-root = hostname foobar
socket = /var/www/foobar.socket
psgi-file = foobar.pl

Linux O_TMPFILE
^^^^^^^^^^^^^^^

Latest Linux kernel support a new operational mode for opening files: O_TMPFILE

this flag open a temporary file (read: unlinked) without any kind of race conditions.

This mode is automatically used if available (no options needed)

Linux pivot-root
^^^^^^^^^^^^^^^^

When dealing with Linux namespaces, changing the root filesystem is one of the main task.

chroot() is generally too simple, while pivot-root allows you more advanced setup

The syntax is ``--pivot-root <new_root> <old_root>``

Cheaper memlimit
^^^^^^^^^^^^^^^^

Author: Łukasz Mierzwa

This new check allows control of dynamic process spawning based on the RSS usage:

https://uwsgi-docs.readthedocs.io/en/latest/Cheaper.html#setting-memory-limits

Log encoders
^^^^^^^^^^^^

There are dozens of log engines and storage system nowadays. The original uWSGI approach was developing a plugin for every engine.

While working with logstash and fluentd we realized that most of the logging pluging are reimplementations of the same concept over and over again.

We followed an even more modular approach introducing log encoders:

:doc:`LogEncoders`

They are basically patterns you can apply to each logline

New "advanced" Hooks
^^^^^^^^^^^^^^^^^^^^

A new series of hooks for developers needing little modifications to the uWSGI cores are available.

Documention about the whole hooks subsystem is now available (it is a work in progress):

:doc:`Hooks`

New mount/umount hooks
^^^^^^^^^^^^^^^^^^^^^^

When dealing with namespaces and jails, mounting and unmounting filesystems is one of the most common tasks.

As the mount and umount commands could not be available during the setup phase, these 2 hooks have been added directly calling the
syscalls.

Check :doc:`Hooks`


Availability
************

uWSGI 1.9.16 has been released on September 14th 2013. You can download it from:

https://projects.unbit.it/downloads/uwsgi-1.9.16.tar.gz

Changelog [20130829]

Bugfixes
^^^^^^^^

* fixed jvm options hashmap (#364)
* fixed python3 wsgi.file_wrapper
* fixed python3 --catch-exceptions
* fixed type in pypy wsgi.input.read
* better symbol detection for pypy
* improved ruby libraries management on heroku
* fixed http-keepalive memleak
* fixed spooler body management under CPython
* fixed unshare() usage of 'fs'
* fixed UWSGI_PROFILE usage when building plugins with --plugin
* improved SmartOS support and added OmniOS support



New features
^^^^^^^^^^^^

The PTY plugin
**************

This new plugin allows you to generate pseudoterminals and attach them to your workers.

Pseudoterminals are then reachable via network (UNIX or TCP sockets).

You can use them for shared debugging or to have input channels on your webapps.

The plugin is in early stage of development (very few features) and it is not built in by default, but you can already make funny things like:

.. code-block:: ini

[uwsgi]
plugin = pty,rack
; generate a new pseudoterminal on port 5000 and map it to the first worker
pty-socket = 127.0.0.1:5000

; classic options
master = true
processes = 2
rack = myapp.ru
socket = /tmp/uwsgi.socket

; run a ruby interactive console (will use the pseudoterminal)
; we use pry as it kick asses
rbshell = require 'pry';binding.pry

now you can access the pseudoterminal with

.. code-block:: sh

uwsgi --plugin pty --pty-connect 127.0.0.1:5000

you can run the client in various windows, it will be shared by all of the peers (all will access the same pseudoterminal).

We are sure new funny uses for it will popup pretty soon

preliminary documentation is available at :doc:`Pty`

strict mode
***********

One of the most common error when writing uWSGI config files, are typos in option names.

As you can add any option in uWSGI config files, the system will accept anythyng you will write even if it is not a real uWSGI option.

While this approach is very powerful and allow lot of funny hacks, it can causes lot of headaches too.

If you want to check all of your options in one step, you can now add the --strict option. Unknown options will trigger a fatal error.

fallback configs
****************

Being very cheap (in term of resources) and supporting lot of operating systems and architectures, uWSGI is heavily used in embedded systems.

One of the common feature in such devices is the "reset to factory defaults".

uWSGI now natively support this kind of operation, thanks to the --fallback-config option.

If a uWSGI instance dies with exit(1) and a fallback-config is specified, the binary will be re-exec()'d with the new config as the only argument.

Let's see an example of a configuration with unbindable address (unprivileged user trying to bind to privileged port)

.. code-block:: ini

[uwsgi]
uid = 1000
gid = 1000
socket = :80

and a fallback one (bind to unprivileged port 8080)

.. code-block:: ini

[uwsgi]
uid = 1000
gid = 1000
socket = :8080

run it (as root, as we want to drop privileges):

.. code-block:: sh

sudo uwsgi --ini wrong.ini --fallback-config right.ini


you will get in your logs:

.. code-block:: sh

...
bind(): Permission denied [core/socket.c line 755]
Thu Aug 29 07:26:26 2013 - !!! /Users/roberta/uwsgi/uwsgi (pid: 12833) exited with status 1 !!!
Thu Aug 29 07:26:26 2013 - !!! Fallback config to right.ini !!!
[uWSGI] getting INI configuration from right.ini
*** Starting uWSGI 1.9.15-dev-4046f76 (64bit) on [Thu Aug 29 07:26:26 2013] ***
...

--perl-exec and --perl-exec-post-fork
*************************************

You can now run custom perl code before and after the fork() calls.

Both options simply take the perl script as the argument

uwsgi.cache_keys([cache])
*************************

This api function has been added to the python and pypy plugins. It allows you to iterate the keys of a local uWSGI cache.

It returns a list.

added `%(ftime)` to logformat
*****************************

this is like 'ltime' but honouring the --log-date format

protect destruction of UNIX sockets when another instance binds them
********************************************************************

on startup uWSGI now get the inode of the just created unix socket.

On vacuum if the inode is changed the unlink of the socket is skipped.

This should help avoiding sysadmin destructive race conditions or misconfigurations

--worker-exec2
**************

this is like --worker-exec but happens after post_fork hooks

allow post_fork hook on general plugins
***************************************

general plugins (the ones without the .request hook) can now expose the .post_fork hook

--call hooks
************

In the same spirit of exec-* hooks, call hooks works in the same way but directly calling functions
in the current process address space (they have to be exposed as valid symbols)

take this c source (call it hello.c):

.. code-block:: c

#include <stdio.h>

void i_am_hello_world_for_uwsgi() {
printf("Hello World!!!\n");
}

and compile it as a shared library:

.. code-block:: sh

gcc -o libhello.so -shared -fPIC hello.c

now choose when (and where) to call it in uWSGI:

.. code-block:: sh

./uwsgi --help | grep -- --call-
--call-pre-jail call the specified function before jailing
--call-post-jail call the specified function after jailing
--call-in-jail call the specified function in jail after initialization
--call-as-root call the specified function before privileges drop
--call-as-user call the specified function after privileges drop
--call-as-user-atexit call the specified function before app exit and reload
--call-pre-app call the specified function before app loading
--call-post-app call the specified function after app loading
--call-as-vassal call the specified function() before exec()ing the vassal
--call-as-vassal1 call the specified function(char *) before exec()ing the vassal
--call-as-vassal3 call the specified function(char *, uid_t, gid_t) before exec()ing the vassal
--call-as-emperor call the specified function() in the emperor after the vassal has been started
--call-as-emperor1 call the specified function(char *) in the emperor after the vassal has been started
--call-as-emperor2 call the specified function(char *, pid_t) in the emperor after the vassal has been started
--call-as-emperor4 call the specified function(char *, pid_t, uid_t, gid_t) in the emperor after the vassal has been started

options ending with a number are variants expecting arguments (the suffix is the number of arguments they take)

we want to call our function just before our application is loaded:

.. code-block:: ini

[uwsgi]
; load the shared library
dlopen = ./libhello.so
; set the hook
call-pre-app = i_am_hello_world_for_uwsgi
...



your custom function will be called just before app loading.

Take in account those functions are called in the process address space, so you can make
all sort of (black) magic with them.

Note: dlopen is a wrapper for the dlopen() function, so all the same rules apply (read: USE ABSOLUTE PATHS !!!)

init_func support for plugins, and --need-plugin variant
********************************************************

when loading a plugin you can call a symbol defined in it soon after dlopen():

.. code-block:: sh

uwsgi --plugin "foobar|myfunc" ...

uWSGI will call the 'myfunc' symbol exposed by the 'foobar' plugin

--need-plugin is like --plugin but will exit(1) the process if plugin loading fails

added commodity loader for the pecan framework
**********************************************

Author: Ryan Petrello

A new python loader (--pecan) has been added for the pecan WSGI framework

http://pecanpy.org/

https://uwsgi-docs.readthedocs.io/en/latest/Python.html#pecan-support

UWSGI_REMOVE_INCLUDES
*********************

during the build phase you can remove include headers with the UWSGI_REMOVE_INCLUDES environment variable.

This is useful for cross-compilation where some automatically detected includes could be wrong

router_expires
**************

We already have various options in the uWSGI core to set Expires header.

This router has been added to allow customizing them:

.. code-block:: ini

[uwsgi]
route = /^foobar1(.*)/ expires:filename=foo$1poo,value=30
route = /^foobar2(.*)/ expires:unix=${time[unix]},value=30

the router takes a filename mtime or a unix time, adds 'value' to it, and return it as an http date.


announce Legion's death on reload/shutdown
******************************************

Every legion member will now announce its death as soon as a reload (or a shutdown) of the instance is triggered

The GlusterFS plugin (beta)
***************************

This new plugin make use ot the new glusterfs c api, avoiding the overhead of fuse when serving files stored on glusterfs servers.

The plugin supports the multiprocess and multithreads modes, while async modes are currently in beta.

Documentation is available: :doc:`GlusterFS`

--force-gateway
***************

all of the gateways (fastrouter, httprouter, rawrouter, sslrouter ...) has to be run under the master process.

By specifying --force-gateway, you will bypass this limit

preliminary python3 profiler (beta)
***********************************

The --profiler pycall/pyline profilers have been added to python3. They are beta quality (they leaks memory), but should be usable.

file monitor support for OpenBSD,NetBSD,DragonFlyBSD
****************************************************

Both --fs-reload and the @fmon decorator now work on this operating systems.

--cwd
*****

you can force the startup "current working directory" (used by --vacuum and the reloading subsystem) with this option.

It is useful in chroot setups where the binary executable change its place.

--add-gid
*********

This options allows you to add additional group ids to the current process. You can specify it multiple times.

Emperor and Linux namespaces improvements
*****************************************

Thanks to the cooperation with the pythonanywhere.com guys the Emperor has been improved for better Linux namespaces integration.

The --emperor-use-clone option allows you to use clone() instead of fork() for your vassal's spawn. In this way you can create the vassals
directly in a new namespace. The function takes the same parameters of the --unshare one

.. code-block:: sh

uwsgi --emperor /etc/vassals --emperor-use-clone pid,uts

will create each vassal in a new pid and uts namespace

while

.. code-block:: sh

uwsgi --emperor /etc/vassals --emperor-use-clone pid,uts,net,ipc,fs

will basically use all of the currently available namespaces.

Two new exec (and call) hooks are available:

--exec-as-emperor will run commands in the emperor soon after a vassal has been spawn (setting 4 env vars, UWSGI_VASSAL_CONFIG, UWSGI_VASSAL_PID, UWSGI_VASSAL_UID and UWSGI_VASSAL_GID)

--exec-as-vassal will run commands in the vassal just before calling exec() (so directly in the new namespaces)


--wait-for-interface
^^^^^^^^^^^^^^^^^^^^

As dealing with the Linux network namespace introduces lot of race conditions (especially when working with virtual ethernets), this new option
allows you to pause an instance until a network interface is available.

This is useful when waiting for the emperor to move a veth to the vassal namespace, avoiding the vassal to run commands on the interface before is available


.. code-block:: ini

[uwsgi]
emperor = /etc/uwsgi/vassals
emperor-use-clone = pid,net,fs,ipc,uts
; each vassal should have its veth pair, so the following commands should be improved
exec-as-emperor = ip link del veth0
exec-as-emperor = ip link add veth0 type veth peer name veth1
; do not use the $(UWSGI_VASSAL_PID) form, otherwise the config parser will expand it on startup !!!
exec-as-emperor = ip link set veth1 netns $UWSGI_VASSAL_PID




.. code-block:: ini

[uwsgi]
; suspend until the emperor attach veth1
wait-for-interface = veth1
; the following hook will be run only after veth1 is available
exec-as-root = hostname vassal001
exec-as-root = ifconfig lo up
exec-as-root = ifconfig veth1 192.168.0.2
uid = vassal001
gid = vassal001
socket = :3031
...


Availability
^^^^^^^^^^^^

uWSGI 1.9.15 has been released on August 29th 2013. You can download it from:

https://projects.unbit.it/downloads/uwsgi-1.9.15.tar.gz

Changelog [20130721]


Bugfixes
********

- fixed python modifier1 management (was hardcoded to 0)
- fixed url decoding in http and http-socket (it now supports lowercase hex, spotted by Miles Shang)
- more user-friendly error message for undeletable unix sockets
- fixed --http-auto-chunked in http 1.1 keepalive mode (André Cruz)
- fixed python wheel support (Fraser Nevett)
- fixed --safe-fd (was not correctly honoured by the Emperor)
- fixed ruby 2.x reloading
- improved support for OSX Tiger (yes, OSX 10.4)
- better computation of listen queue load
- fixed v8 build on OSX
- fixed pypy rpc
- improved chunked api performance
- fixed latin1 encoding with python3
- fixed --spooler-ordered (Roberto Leandrini)
- fixed php status line reported in request logs


New features
************

Ruby 1.9.x/2.x native threads support
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Ruby 1.9 (mri) introduced native threads support (very similar to the CPython ones, governed by a global lock named GVL).

For various reasons (check the comments on top of the source plugin) the ruby threads support in uWSGI has been implemented as a "loop engine plugin".

You need to build the "rbthreads" plugin (it is automatic when using the 'ruby2' build profile) and enable it with '--rbthreads'

The gem script has been extended, automatically selecting the 'ruby2' build profile when a ruby >= 1.9 is detected (this should make the life easier for Heroku users)

Rails4 is the first Ruby on Rails version supporting and blessing threads (in 3.x you need to explicitly enable support). You can use
multiple threads in Rails4 only when in "production" mode, otherwise your app will deadlock after the first request.

An example config:

.. code-block:: ini

[uwsgi]
plugins = rack,rbthreads
master = true
; spawn 2 processes
processes = 2
; spawn 8 threads
threads = 8
; enable ruby threads
rbthreads = true
; load the Rack app
rack = config.ru
; bind to an http port
http-socket = :9090
http-socket-modifier1 = 7

it will generate a total of 16 threads

Filesystem monitoring interface (fsmon)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Currently uWSGI is able to monitor filesystem changes using the "simple" --touch-* facility or the signal framework (using various
operating system api like inotify or kqueue).

A new interface for plugin writers named "fsmon" has been added, allowing easy implementation of realtime filesystem monitors.

Three new options have been added:

`--fs-reload <path>`


`--fs-brutal-reload <path>`


`--fs-signal <path> <signal>`

Contrary to the --touch-* options they are realtime (the master is woke up as soon as the item changes) and. uses kernel facilities
(currently only inotify() and kqueue() are supported). Thanks to this choice you can now monitor a whole directory for changes (without the need of external
processes/wrapper like inotifywatch)

uClibc support
^^^^^^^^^^^^^^

Author: Natanael Copa

uWSGI can now be built on uclibc-based systems (generally, embedded systems)

Alpine Linux is the operating system on which the support has been tested

Lua 5.2 support
^^^^^^^^^^^^^^^

Author: Natanael Copa

the lua plugins now supports Lua 5.2

setscheme, setdocroot
^^^^^^^^^^^^^^^^^^^^^

This two new routing actions allow you to dynamically override DOCUMENT_ROOT and UWSGI_SCHEME

sendfile, fastfile
^^^^^^^^^^^^^^^^^^

This two actions (added to the router_static plugin) allows you to return static files to the client bypassing the DOCUMENT_ROOT check.

The first one forces the use of the sendfile() syscall (where available), while the second automatically tries to choose the best serving strategy (like offloading)

--reload-on-fd and --brutal-reload-on-fd
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Two new options allowing you to reload an instance when a file descriptor is ready.

Currently the best usage scenario is for the oom_control cgroup interface (via eventfd).

Supposing you have a process wrapper allocating an eventfd() reporting OOM events (and exposed as the 'OOM' environment var) you can force a uWSGI reload
when out of memory with:

.. code-block:: ini

[uwsgi]
...
reload-on-fd = $(OOM):8 OUT OF MEMORY !!!


it means:

monitor the $(OOM) file descriptor and read 8 bytes from it when ready (it is an eventfd() requirement), then print "OUT OF MEMORY !!!" in the logs and gracefully reload the instance.

Obviously this is only a way to use it. The UNIX world is file-descriptor based so you have plenty of funny ways to use it.


Spooler improvements
^^^^^^^^^^^^^^^^^^^^

Author: Roberto Leandrini


Effectively all of the work has been done in uwsgidecorators.py

You can now pass to all of the available spooler-related decorators the "pass_arguments=True" option, to automatically
serialize the spooler function parameters. This is an abstraction avoiding you the need to serialize/deserialize arguments.

In addition to this the decorators have been extended to implement __call__ in this way you can directly call spooler decorated functions
as normal functions.

--emperor-nofollow
^^^^^^^^^^^^^^^^^^

Enabling this option will allows the Emperor to watch for symbolic links mtime update instead of the mtime of the real file.

Alberto Scotto is working on an updated version supporting both (should be ready for the next release)

daemontools envdir support
^^^^^^^^^^^^^^^^^^^^^^^^^^

Albeit daemontools look old-fashioned, things like envdirs (http://cr.yp.to/daemontools/envdir.html) are heavily used in various context.

uWSGI got two new options (--envdir <path> and --early-envdir <path>) allowing you to support this special (archaic ?) configuration way.

xmldir improvements
^^^^^^^^^^^^^^^^^^^

Author: Guido Berhoerster

The xmldir plugins has been improved supporting iconv-based utf8 encoding. Various minor fixes have been committed.

The examples directory contains two new files showing an xmldir+xslt usage


Breaking News !!!
*****************

Servlet 2.5 support development has just started. The plugin is present in the tree but it is unusable (it is a hardcoded
jsp engine). We expect a beta version after the summer. Obviously we shameless consider :doc:`JWSGI` a better approach than servlet for non-Enterprise people ;)

Availability
************

Download uWSGI 1.9.14 from

https://projects.unbit.it/downloads/uwsgi-1.9.14.tar.gz