--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/src/source/zodb.rst Thu Dec 06 08:24:10 2018 +0100
@@ -0,0 +1,209 @@
+.. _zodb:
+
+
+ZODB server
+============
+
+
+Installation
+''''''''''''
+
+PyAMS relies mainly on a ZODB (**\Z**\ope **\O**\bjects **\D**\ata\ **\B**\ase) to store it's configuration and for exemple *PyAMS_content*, but other packages may
+rely on other database(s).
+
+Concurrent accesses are **required** on the ZODB, to fully enjoy PyAMS.
+Several ZODB storages implementations providing a shared access are available and the installation are describe below:
+
+ * `ZEO Server`_
+ * `RelStorage`_
+ * `Newt.DB`_
+
+
+.. note::
+
+ When you start your application in "*single process*" mode, PyAMS packages start several processes
+ and the "*synchronization*" is done via **ØMQ**.
+
+
+
+.. _`ZEO Server`:
+
+1. ZEO server
++++++++++++++
+
+ZEO (**\Z**\ope **\E**\nterprise **\O**\bjects) is the first available implementation available for concurrent access to a
+FileStorage, provided through the ZEO package.
+
+.. seealso::
+
+ ZEO package documentation and complete configuration settings are available on PyPI_ (cf: `ZOE 5.2.0`_).
+
+.. _PyPI: https://pypi.python.org/pypi/ZEO
+.. _`ZOE 5.2.0`: https://pypi.python.org/pypi/ZEO/5.2.0
+
+
+a) Installation with initial buildout
+-------------------------------------
+
+PyAMS provides a ZEO server scaffold, called *zeo_server*, generated via a *cookiecutter* template.
+
+A simple option to create a ZEO server is to create a buildout environment including *ZEO* and *ZODB* packages:
+
+.. code-block:: bash
+
+ $ mkdir /var/local/
+ $ pip3 install virtualenv
+ $ virtualenv --python=python3.5 env
+ $ cd env
+ $ . bin/activate
+ (env) $ pip3.5 install cookiecutter
+ (env) $ cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server
+
+*CookieCutter* will ask you for a small set of input variables that you can change or not:
+
+- **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release;
+ you can also choose to point to a given release ("0.1.4" for example)
+
+- **project_name**: current environment name in "human form"
+
+- **project_slug**: "technical" package name, based on project name
+
+- **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be
+ shared with other projects ("eggs" as default)
+
+- **run_user**: user name under which ZEO process will run ("zeo" as default)
+
+- **run_group**: group name under which ZEO process will run ("zeo" as default)
+
+- **zeo_server_port**: listening port of ZEO server ("8100" as default)
+
+- **zeo_storage**: name of first ZEO storage ("pyams" as default)
+
+- **zeo_pack_report**: email address to which pack reports should be sent
+
+- **logs_directory**: absolute path to directory containing ZEO's log files.
+
+A message is displayed after initialization to finalize environment creation:
+
+.. code::
+
+ Your ZEO environment is initialized.
+ To finalize it''s creation, just type:
+ - cd zeo_server
+ - python3.5 bootstrap.py
+ - ./bin/buildout
+
+ To initialize authentication database, please run following command after buildout:
+ ./bin/zeopasswd -f etc/auth.db -p digest -r "ZEO_server" zeouser xxxx
+
+b) ZEO server configuration
+---------------------------
+
+All ZEO configuration files are generated in "etc" subdirectory. These includes:
+
+- **etc/zeo_server-zdaemon.conf**: ZDaemon configuration file
+
+- **etc/zeo_server-zeo.conf**: ZEO server configuration file
+
+- **etc/auth.db**: ZEO authentication file; WARNING: this file is not created automatically, you have to create it
+ after buildout.
+
+In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during
+*CookieCutter* template creation.
+
+
+c) ZEO server tools
+-------------------
+
+A set of system configuration files are produced to handle your ZEO environment. These includes:
+
+- **etc/init.d/zeo-zeo_server**: ZEO server start/stop script in Init-D format. Create a link to this file in
+ */etc/init.d* and update Init.d scripts (*update-rc.d zeo-zeo_server defaults*) to include ZEO in server start/stop
+ process. You can also use this script to start/stop ZEO by hand with *start* and *stop* arguments.
+
+- **etc/systemd/zeo-zeo_server.service**: SystemD service configuration file for ZEO server. Create a link to this
+ file in */etc/systemd/system* and reload SystemD daemon (*systemctl daemon-reload*) before activating ZEO service
+ (*systemctl enable zeo-zeo_server.service* and *systemctl start zeo-zeo_server.service*).
+
+- **etc/logrotate.d/zeo-zeo_server**: LogRotate configuration file for ZEO log files. Create a link to this file in
+ */etc/logrotate.d* to activate log rotation for ZEO server.
+
+- **etc/cron.d/pack-zeo-zeo_server**: Cron configuration file for ZEO database packing. Just create a link to this
+ file in */etc/cron.d* directory to enable ZODB packing on a weekly basis (by default).
+
+In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during
+*CookieCutter* template creation. All directory names are those used on a Debian GNU/Linux distribution and may have
+to be changed on other distributions.
+
+
+.. _RelStorage:
+
+2. RelStorage server
+++++++++++++++++++++
+
+RelStorage (http://relstorage.readthedocs.io/en/latest) is an alternate ZODB storage implementation, that stores
+Python pickles in a relational database; PostgreSQL (>= 9.0), MySQL (>= 5.0.32) and Oracle (> 10g) databases are
+supported.
+
+To create a database compatible with RelStorage, you just have to install the database server and create a database
+dedicated to RelStorage; schema initialization is then completely done by RelStorage on application startup.
+
+RelStorage is supposed to provide better performances than ZEO, notably under high load. RelStorage can also get
+benefit from many extensions (clustering, fail-over, hot-standby...) provided by these databases.
+
+
+.. _Newt.DB:
+
+3. NewtDB server
+++++++++++++++++
+
+NewtDB (http://www.newtdb.org/en/latest) is another ZODB storage implementation. It's using RelStorage but is
+dedicated to PostgreSQL (>= 9.5).
+
+NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB
+column. The JSON data supplements the native data to support indexing, search, and access from non-Python application.
+Because the JSON format is lossy, compared to the native format, the native format is still used for loading
+objects from the database. For this reason, the JSON data are read-only.
+
+Newt adds a search API for searching the Postgres JSON data and returning persistent objects. It also provides a
+convenient API for raw data searches.
+
+Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available
+if you need to switch from a classic RelStorage database to a Newt database.
+
+
+.. _ZODB.migration:
+
+Migration
+'''''''''
+
+After installation, you can switch from a given storage to another one with the help of the *zodbconvert* command
+line script provided by RelStorage.
+
+This Python script is using a configuration file containing directives of both source and target databases, which can
+be any storage described in the previous sections (or can even use the same storage).
+
+Here is a sample configuration file to convert a ZODB from a ZEO to RelStorage:
+
+
+.. code-block:: html
+
+ %import relstorage
+
+ <zeoclient source>
+ server zeo-server.mydomain:8100
+ storage pyams
+ blob-dir /var/local/env/zeo/var/zeoclient/blobs
+ shared-blob-dir false
+ </zeoclient>
+
+ <relstorage destination>
+ keep-history false
+ blob-dir /var/local/env/pyams/var/relstorage/blobs
+ shared-blob-dir false
+ <postgresql>
+ dsn host='postgresql-server.mydomain' dbname='pyams_rs' user='pyams' password='xxxxxxxx'
+ </postgresql>
+ </relstorage>
+
+