src/source/zodb.rst
changeset 0 d153941bb745
child 10 1d12ff3f036a
equal deleted inserted replaced
-1:000000000000 0:d153941bb745
       
     1 .. _zodb:
       
     2 
       
     3 Creating ZODB
       
     4 =============
       
     5 
       
     6 PyAMS primarily relies on a ZODB (Zope Objects DataBase) to store it's configuration. Other packages may
       
     7 rely on other database(s), but *PyAMS_content* package also stores it's contents in a ZODB.
       
     8 
       
     9 As some PyAMS packages start several processes ("synchronization" is done via **ØMQ**), concurrent accesses are
       
    10 required on the ZODB (even when you start your application in "single process" mode); several ZODB storages
       
    11 implementations providing a shared access are available: `ZEO`_, `RelStorage`_ and `Newt.DB`_.
       
    12 
       
    13 
       
    14 .. _ZEO:
       
    15 
       
    16 Installing a ZEO server
       
    17 +++++++++++++++++++++++
       
    18 
       
    19 ZEO (Zope Enterprise Objects) is the first available implementation available for concurrent access to a
       
    20 FileStorage, provided through the ZEO package.
       
    21 
       
    22 ZEO package documentation and complete configuration settings are available on PyPI.
       
    23 
       
    24 Creating initial buildout
       
    25 -------------------------
       
    26 
       
    27 PyAMS provides a ZEO server scaffold, called *zeo_server*, generated via a *cookiecutter* template.
       
    28 
       
    29 A simple option to create a ZEO server is to create a buildout environment including *ZEO* and *ZODB* packages:
       
    30 
       
    31 .. code-block:: bash
       
    32 
       
    33     # mkdir /var/local/
       
    34     # pip3 install virtualenv
       
    35     # virtualenv --python=python3.5 env
       
    36     # cd env
       
    37     # . bin/activate
       
    38     (env) # pip3.5 install cookiecutter
       
    39     (env) # cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server
       
    40 
       
    41 *CookieCutter* will ask you for a small set of input variables that you can change or not:
       
    42 
       
    43 - **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release;
       
    44   you can also choose to point to a given release ("0.1.4" for example)
       
    45 
       
    46 - **project_name**: current environment name in "human form"
       
    47 
       
    48 - **project_slug**: "technical" package name, based on project name
       
    49 
       
    50 - **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be
       
    51   shared with other projects ("eggs" as default)
       
    52 
       
    53 - **run_user**: user name under which ZEO process will run ("zeoadm" as default)
       
    54 
       
    55 - **run_group**: group name under which ZEO process will run ("zeo" as default)
       
    56 
       
    57 - **zeo_server_port**: listening port of ZEO server ("8100" as default)
       
    58 
       
    59 - **zeo_monitor_port**: listening port of ZEO monitor ("8101" as default)
       
    60 
       
    61 - **zeo_storage**: name of first ZEO storage; default value is based on project name
       
    62 
       
    63 - **use_zeo_auth**: specify if ZEO authentication should be used
       
    64 
       
    65 - **zeo_auth_user**: name of ZEO authenticated user (if ZEO authentication is used)
       
    66 
       
    67 - **zeo_auth_password**: password of ZEO authenticated user (if ZEO authentication is used)
       
    68 
       
    69 - **zeo_pack_report**: email address to which pack reports should be sent
       
    70 
       
    71 - **logs_directory**: absolute path to directory containing ZEO's log files.
       
    72 
       
    73 A message is displayed after initialization to finalize environment creation:
       
    74 
       
    75 .. code::
       
    76 
       
    77     Your ZEO environment is initialized.
       
    78     To finalize it''s creation, just type:
       
    79     - cd zeo_server
       
    80     - python3.5 bootstrap.py
       
    81     - ./bin/buildout
       
    82 
       
    83     To initialize authentication database, please run following command after buildout:
       
    84     ./bin/zeopasswd -f etc/auth.db -p digest -r "ZEO_server" zeouser xxxx
       
    85 
       
    86 ZEO server configuration
       
    87 ------------------------
       
    88 
       
    89 All ZEO configuration files are generated in "etc" subdirectory. These includes:
       
    90 
       
    91 - **etc/zeo_server-zdaemon.conf**: ZDaemon configuration file
       
    92 
       
    93 - **etc/zeo_server-zeo.conf**: ZEO server configuration file
       
    94 
       
    95 - **etc/auth.db**: ZEO authentication file; WARNING: this file is not created automatically, you have to create it
       
    96   after buildout.
       
    97 
       
    98 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during
       
    99 *CookieCutter* template creation.
       
   100 
       
   101 ZEO server tools
       
   102 ----------------
       
   103 
       
   104 A set of system configuration files are produced to handle your ZEO environment. These includes:
       
   105 
       
   106 - **etc/init.d/zeo-zeo_server**: ZEO server start/stop script in Init-D format. Create a link to this file in
       
   107   */etc/init.d* and update Init.d scripts (*update-rc.d zeo-zeo_server defaults*) to include ZEO in server start/stop
       
   108   process. You can also use this script to start/stop ZEO by hand with *start* and *stop* arguments.
       
   109 
       
   110 - **etc/systemd/zeo-zeo_server.service**: SystemD service configuration file for ZEO server. Create a link to this
       
   111   file in */etc/systemd/system* and reload SystemD daemon (*systemctl daemon-reload*) before activating ZEO service
       
   112   (*systemctl enable zeo-zeo_server.service* and *systemctl start zeo-zeo_server.service*).
       
   113 
       
   114 - **etc/logrotate.d/zeo-zeo_server**: LogRotate configuration file for ZEO log files. Create a link to this file in
       
   115   */etc/logrotate.d* to activate log rotation for ZEO server.
       
   116 
       
   117 - **etc/cron.d/pack-zeo-zeo_server**: Cron configuration file for ZEO database packing. Just create a link to this
       
   118   file in */etc/cron.d* directory to enable ZODB packing on a weekly basis (by default).
       
   119 
       
   120 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during
       
   121 *CookieCutter* template creation. All directory names are those used on a Debian GNU/Linux distribution and may have
       
   122 to be changed on other distributions.
       
   123 
       
   124 
       
   125 .. _RelStorage:
       
   126 
       
   127 Installing a RelStorage server
       
   128 ++++++++++++++++++++++++++++++
       
   129 
       
   130 RelStorage (http://relstorage.readthedocs.io/en/latest) is an alternate ZODB storage implementation, that stores
       
   131 Python pickles in a relational database; PostgreSQL (>= 9.0), MySQL (>= 5.0.32) and Oracle (> 10g) databases are
       
   132 supported.
       
   133 
       
   134 To create a database compatible with RelStorage, you just have to install the database server and create a database
       
   135 dedicated to RelStorage; schema initialization is then completely done by RelStorage on application startup.
       
   136 
       
   137 RelStorage is supposed to provide better performances than ZEO, notably under high load. RelStorage can also get
       
   138 benefit from many extensions (clustering, fail-over, hot-standby...) provided by these databases.
       
   139 
       
   140 
       
   141 .. _Newt.DB:
       
   142 
       
   143 Installing a NewtDB server
       
   144 ++++++++++++++++++++++++++
       
   145 
       
   146 NewtDB (http://www.newtdb.org/en/latest) is another ZODB storage implementation. It's using RelStorage but is
       
   147 dedicated to PostgreSQL (>= 9.5).
       
   148 
       
   149 NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB
       
   150 column. The JSON data supplements the native data to support indexing, search, and access from non-Python application.
       
   151 Because the JSON format is lossy, compared to the native format, the native format is still used for loading
       
   152 objects from the database. For this reason, the JSON data are read-only.
       
   153 
       
   154 Newt adds a search API for searching the Postgres JSON data and returning persistent objects. It also provides a
       
   155 convenient API for raw data searches.
       
   156 
       
   157 Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available
       
   158 if you need to switch from a classic RelStorage database to a Newt database.