src/source/zodb.rst
branchdev-dc
changeset 34 f234f01d54f9
parent 21 e1cb9c606859
child 104 942151432421
equal deleted inserted replaced
33:eeef4052fcd1 34:f234f01d54f9
     1 .. _zodb:
     1 .. _zodb:
     2 
     2 
     3 Creating ZODB
     3 
     4 =============
     4 ZODB server
     5 
     5 ============
     6 PyAMS primarily relies on a ZODB (Zope Objects DataBase) to store it's configuration. Other packages may
     6 
     7 rely on other database(s), but *PyAMS_content* package also stores it's contents in a ZODB.
     7 
     8 
     8 Installation
     9 As some PyAMS packages start several processes ("synchronization" is done via **ØMQ**), concurrent accesses are
     9 ''''''''''''
    10 required on the ZODB (even when you start your application in "single process" mode); several ZODB storages
    10 
    11 implementations providing a shared access are available: `ZEO`_, `RelStorage`_ and `Newt.DB`_.
    11 PyAMS relies mainly on a ZODB (**\Z**\ope **\O**\bjects **\D**\ata\ **\B**\ase) to store it's configuration and for exemple *PyAMS_content*, but other packages may
    12 
    12 rely on other database(s).
    13 
    13 
    14 .. _ZEO:
    14 Concurrent accesses are **required** on the ZODB, to fully enjoy PyAMS.
    15 
    15 Several ZODB storages implementations providing a shared access are available and the installation are describe below:
    16 Installing a ZEO server
    16 
    17 +++++++++++++++++++++++
    17     * `ZEO Server`_
    18 
    18     * `RelStorage`_
    19 ZEO (Zope Enterprise Objects) is the first available implementation available for concurrent access to a
    19     * `Newt.DB`_
       
    20 
       
    21 
       
    22 .. note::
       
    23 
       
    24     When you start your application in "*single process*" mode, PyAMS packages start several processes
       
    25     and the "*synchronization*" is done via **ØMQ**.
       
    26 
       
    27 
       
    28 
       
    29 .. _`ZEO Server`:
       
    30 
       
    31 1. ZEO server
       
    32 +++++++++++++
       
    33 
       
    34 ZEO (**\Z**\ope **\E**\nterprise **\O**\bjects) is the first available implementation available for concurrent access to a
    20 FileStorage, provided through the ZEO package.
    35 FileStorage, provided through the ZEO package.
    21 
    36 
    22 ZEO package documentation and complete configuration settings are available on PyPI.
    37 .. seealso::
    23 
    38 
    24 Creating initial buildout
    39     ZEO package documentation and complete configuration settings are available on PyPI_ (cf: `ZOE 5.2.0`_).
    25 -------------------------
    40 
       
    41 .. _PyPI: https://pypi.python.org/pypi/ZEO
       
    42 .. _`ZOE 5.2.0`: https://pypi.python.org/pypi/ZEO/5.2.0
       
    43 
       
    44 
       
    45 a) Installation with initial buildout
       
    46 -------------------------------------
    26 
    47 
    27 PyAMS provides a ZEO server scaffold, called *zeo_server*, generated via a *cookiecutter* template.
    48 PyAMS provides a ZEO server scaffold, called *zeo_server*, generated via a *cookiecutter* template.
    28 
    49 
    29 A simple option to create a ZEO server is to create a buildout environment including *ZEO* and *ZODB* packages:
    50 A simple option to create a ZEO server is to create a buildout environment including *ZEO* and *ZODB* packages:
    30 
    51 
    31 .. code-block:: bash
    52 .. code-block:: bash
    32 
    53 
    33     # mkdir /var/local/
    54     $ mkdir /var/local/
    34     # pip3 install virtualenv
    55     $ pip3 install virtualenv
    35     # virtualenv --python=python3.5 env
    56     $ virtualenv --python=python3.5 env
    36     # cd env
    57     $ cd env
    37     # . bin/activate
    58     $ . bin/activate
    38     (env) # pip3.5 install cookiecutter
    59     (env) $ pip3.5 install cookiecutter
    39     (env) # cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server
    60     (env) $ cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server
    40 
    61 
    41 *CookieCutter* will ask you for a small set of input variables that you can change or not:
    62 *CookieCutter* will ask you for a small set of input variables that you can change or not:
    42 
    63 
    43 - **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release;
    64 - **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release;
    44   you can also choose to point to a given release ("0.1.4" for example)
    65   you can also choose to point to a given release ("0.1.4" for example)
    73     - ./bin/buildout
    94     - ./bin/buildout
    74 
    95 
    75     To initialize authentication database, please run following command after buildout:
    96     To initialize authentication database, please run following command after buildout:
    76     ./bin/zeopasswd -f etc/auth.db -p digest -r "ZEO_server" zeouser xxxx
    97     ./bin/zeopasswd -f etc/auth.db -p digest -r "ZEO_server" zeouser xxxx
    77 
    98 
    78 ZEO server configuration
    99 b) ZEO server configuration
    79 ------------------------
   100 ---------------------------
    80 
   101 
    81 All ZEO configuration files are generated in "etc" subdirectory. These includes:
   102 All ZEO configuration files are generated in "etc" subdirectory. These includes:
    82 
   103 
    83 - **etc/zeo_server-zdaemon.conf**: ZDaemon configuration file
   104 - **etc/zeo_server-zdaemon.conf**: ZDaemon configuration file
    84 
   105 
    88   after buildout.
   109   after buildout.
    89 
   110 
    90 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during
   111 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during
    91 *CookieCutter* template creation.
   112 *CookieCutter* template creation.
    92 
   113 
    93 ZEO server tools
   114 
    94 ----------------
   115 c) ZEO server tools
       
   116 -------------------
    95 
   117 
    96 A set of system configuration files are produced to handle your ZEO environment. These includes:
   118 A set of system configuration files are produced to handle your ZEO environment. These includes:
    97 
   119 
    98 - **etc/init.d/zeo-zeo_server**: ZEO server start/stop script in Init-D format. Create a link to this file in
   120 - **etc/init.d/zeo-zeo_server**: ZEO server start/stop script in Init-D format. Create a link to this file in
    99   */etc/init.d* and update Init.d scripts (*update-rc.d zeo-zeo_server defaults*) to include ZEO in server start/stop
   121   */etc/init.d* and update Init.d scripts (*update-rc.d zeo-zeo_server defaults*) to include ZEO in server start/stop
   114 to be changed on other distributions.
   136 to be changed on other distributions.
   115 
   137 
   116 
   138 
   117 .. _RelStorage:
   139 .. _RelStorage:
   118 
   140 
   119 Installing a RelStorage server
   141 2. RelStorage server
   120 ++++++++++++++++++++++++++++++
   142 ++++++++++++++++++++
   121 
   143 
   122 RelStorage (http://relstorage.readthedocs.io/en/latest) is an alternate ZODB storage implementation, that stores
   144 RelStorage (http://relstorage.readthedocs.io/en/latest) is an alternate ZODB storage implementation, that stores
   123 Python pickles in a relational database; PostgreSQL (>= 9.0), MySQL (>= 5.0.32) and Oracle (> 10g) databases are
   145 Python pickles in a relational database; PostgreSQL (>= 9.0), MySQL (>= 5.0.32) and Oracle (> 10g) databases are
   124 supported.
   146 supported.
   125 
   147 
   130 benefit from many extensions (clustering, fail-over, hot-standby...) provided by these databases.
   152 benefit from many extensions (clustering, fail-over, hot-standby...) provided by these databases.
   131 
   153 
   132 
   154 
   133 .. _Newt.DB:
   155 .. _Newt.DB:
   134 
   156 
   135 Installing a NewtDB server
   157 3. NewtDB server
   136 ++++++++++++++++++++++++++
   158 ++++++++++++++++
   137 
   159 
   138 NewtDB (http://www.newtdb.org/en/latest) is another ZODB storage implementation. It's using RelStorage but is
   160 NewtDB (http://www.newtdb.org/en/latest) is another ZODB storage implementation. It's using RelStorage but is
   139 dedicated to PostgreSQL (>= 9.5).
   161 dedicated to PostgreSQL (>= 9.5).
   140 
   162 
   141 NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB
   163 NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB
   148 
   170 
   149 Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available
   171 Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available
   150 if you need to switch from a classic RelStorage database to a Newt database.
   172 if you need to switch from a classic RelStorage database to a Newt database.
   151 
   173 
   152 
   174 
   153 .. _ZODB.init:
       
   154 
       
   155 Initializing ZODB
       
   156 +++++++++++++++++
       
   157 
       
   158 When your ZODB is created, you have to initialize it!
       
   159 
       
   160 After installing PyAMS application (see :ref:`appinstall`), a database upgrade script is available which allows you
       
   161 to initialize the database. See :ref:`config` and :ref:`scripts` to know more.
       
   162 
       
   163 
       
   164 .. _ZODB.migration:
   175 .. _ZODB.migration:
   165 
   176 
   166 ZODB migration
   177 Migration
   167 ++++++++++++++
   178 '''''''''
   168 
   179 
   169 After installation, you can switch from a given storage to another one with the help of the *zodbconvert* command
   180 After installation, you can switch from a given storage to another one with the help of the *zodbconvert* command
   170 line script provided by RelStorage.
   181 line script provided by RelStorage.
   171 
   182 
   172 This Python script is using a configuration file containing directives of both source and target databases, which can
   183 This Python script is using a configuration file containing directives of both source and target databases, which can
   173 be any storage described in the previous sections (or can even use the same storage).
   184 be any storage described in the previous sections (or can even use the same storage).
   174 
   185 
   175 Here is a sample configuration file to convert a ZODB from a ZEO to RelStorage:
   186 Here is a sample configuration file to convert a ZODB from a ZEO to RelStorage:
   176 
   187 
   177 .. code-block::
   188 
       
   189 .. code-block:: html
   178 
   190 
   179     %import relstorage
   191     %import relstorage
   180 
   192 
   181     <zeoclient source>
   193     <zeoclient source>
   182         server zeo-server.mydomain:8100
   194         server zeo-server.mydomain:8100
   191         shared-blob-dir false
   203         shared-blob-dir false
   192         <postgresql>
   204         <postgresql>
   193             dsn host='postgresql-server.mydomain' dbname='pyams_rs' user='pyams' password='xxxxxxxx'
   205             dsn host='postgresql-server.mydomain' dbname='pyams_rs' user='pyams' password='xxxxxxxx'
   194         </postgresql>
   206         </postgresql>
   195     </relstorage>
   207     </relstorage>
       
   208 
       
   209