|
1 .. _zodb: |
|
2 |
|
3 |
|
4 ZODB server |
|
5 ============ |
|
6 |
|
7 |
|
8 Installation |
|
9 '''''''''''' |
|
10 |
|
11 PyAMS relies mainly on a ZODB (**\Z**\ope **\O**\bjects **\D**\ata\ **\B**\ase) to store it's configuration and for exemple *PyAMS_content*, but other packages may |
|
12 rely on other database(s). |
|
13 |
|
14 Concurrent accesses are **required** on the ZODB, to fully enjoy PyAMS. |
|
15 Several ZODB storages implementations providing a shared access are available and the installation are describe below: |
|
16 |
|
17 * `ZEO Server`_ |
|
18 * `RelStorage`_ |
|
19 * `Newt.DB`_ |
|
20 |
|
21 |
|
22 .. note:: |
|
23 |
|
24 When you start your application in "*single process*" mode, PyAMS packages start several processes |
|
25 and the "*synchronization*" is done via **ØMQ**. |
|
26 |
|
27 |
|
28 |
|
29 .. _`ZEO Server`: |
|
30 |
|
31 1. ZEO server |
|
32 +++++++++++++ |
|
33 |
|
34 ZEO (**\Z**\ope **\E**\nterprise **\O**\bjects) is the first available implementation available for concurrent access to a |
|
35 FileStorage, provided through the ZEO package. |
|
36 |
|
37 .. seealso:: |
|
38 |
|
39 ZEO package documentation and complete configuration settings are available on PyPI_ (cf: `ZOE 5.2.0`_). |
|
40 |
|
41 .. _PyPI: https://pypi.python.org/pypi/ZEO |
|
42 .. _`ZOE 5.2.0`: https://pypi.python.org/pypi/ZEO/5.2.0 |
|
43 |
|
44 |
|
45 a) Installation with initial buildout |
|
46 ------------------------------------- |
|
47 |
|
48 PyAMS provides a ZEO server scaffold, called *zeo_server*, generated via a *cookiecutter* template. |
|
49 |
|
50 A simple option to create a ZEO server is to create a buildout environment including *ZEO* and *ZODB* packages: |
|
51 |
|
52 .. code-block:: bash |
|
53 |
|
54 $ mkdir /var/local/ |
|
55 $ pip3 install virtualenv |
|
56 $ virtualenv --python=python3.5 env |
|
57 $ cd env |
|
58 $ . bin/activate |
|
59 (env) $ pip3.5 install cookiecutter |
|
60 (env) $ cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server |
|
61 |
|
62 *CookieCutter* will ask you for a small set of input variables that you can change or not: |
|
63 |
|
64 - **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release; |
|
65 you can also choose to point to a given release ("0.1.4" for example) |
|
66 |
|
67 - **project_name**: current environment name in "human form" |
|
68 |
|
69 - **project_slug**: "technical" package name, based on project name |
|
70 |
|
71 - **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be |
|
72 shared with other projects ("eggs" as default) |
|
73 |
|
74 - **run_user**: user name under which ZEO process will run ("zeo" as default) |
|
75 |
|
76 - **run_group**: group name under which ZEO process will run ("zeo" as default) |
|
77 |
|
78 - **zeo_server_port**: listening port of ZEO server ("8100" as default) |
|
79 |
|
80 - **zeo_storage**: name of first ZEO storage ("pyams" as default) |
|
81 |
|
82 - **zeo_pack_report**: email address to which pack reports should be sent |
|
83 |
|
84 - **logs_directory**: absolute path to directory containing ZEO's log files. |
|
85 |
|
86 A message is displayed after initialization to finalize environment creation: |
|
87 |
|
88 .. code:: |
|
89 |
|
90 Your ZEO environment is initialized. |
|
91 To finalize it''s creation, just type: |
|
92 - cd zeo_server |
|
93 - python3.5 bootstrap.py |
|
94 - ./bin/buildout |
|
95 |
|
96 To initialize authentication database, please run following command after buildout: |
|
97 ./bin/zeopasswd -f etc/auth.db -p digest -r "ZEO_server" zeouser xxxx |
|
98 |
|
99 b) ZEO server configuration |
|
100 --------------------------- |
|
101 |
|
102 All ZEO configuration files are generated in "etc" subdirectory. These includes: |
|
103 |
|
104 - **etc/zeo_server-zdaemon.conf**: ZDaemon configuration file |
|
105 |
|
106 - **etc/zeo_server-zeo.conf**: ZEO server configuration file |
|
107 |
|
108 - **etc/auth.db**: ZEO authentication file; WARNING: this file is not created automatically, you have to create it |
|
109 after buildout. |
|
110 |
|
111 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during |
|
112 *CookieCutter* template creation. |
|
113 |
|
114 |
|
115 c) ZEO server tools |
|
116 ------------------- |
|
117 |
|
118 A set of system configuration files are produced to handle your ZEO environment. These includes: |
|
119 |
|
120 - **etc/init.d/zeo-zeo_server**: ZEO server start/stop script in Init-D format. Create a link to this file in |
|
121 */etc/init.d* and update Init.d scripts (*update-rc.d zeo-zeo_server defaults*) to include ZEO in server start/stop |
|
122 process. You can also use this script to start/stop ZEO by hand with *start* and *stop* arguments. |
|
123 |
|
124 - **etc/systemd/zeo-zeo_server.service**: SystemD service configuration file for ZEO server. Create a link to this |
|
125 file in */etc/systemd/system* and reload SystemD daemon (*systemctl daemon-reload*) before activating ZEO service |
|
126 (*systemctl enable zeo-zeo_server.service* and *systemctl start zeo-zeo_server.service*). |
|
127 |
|
128 - **etc/logrotate.d/zeo-zeo_server**: LogRotate configuration file for ZEO log files. Create a link to this file in |
|
129 */etc/logrotate.d* to activate log rotation for ZEO server. |
|
130 |
|
131 - **etc/cron.d/pack-zeo-zeo_server**: Cron configuration file for ZEO database packing. Just create a link to this |
|
132 file in */etc/cron.d* directory to enable ZODB packing on a weekly basis (by default). |
|
133 |
|
134 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during |
|
135 *CookieCutter* template creation. All directory names are those used on a Debian GNU/Linux distribution and may have |
|
136 to be changed on other distributions. |
|
137 |
|
138 |
|
139 .. _RelStorage: |
|
140 |
|
141 2. RelStorage server |
|
142 ++++++++++++++++++++ |
|
143 |
|
144 RelStorage (http://relstorage.readthedocs.io/en/latest) is an alternate ZODB storage implementation, that stores |
|
145 Python pickles in a relational database; PostgreSQL (>= 9.0), MySQL (>= 5.0.32) and Oracle (> 10g) databases are |
|
146 supported. |
|
147 |
|
148 To create a database compatible with RelStorage, you just have to install the database server and create a database |
|
149 dedicated to RelStorage; schema initialization is then completely done by RelStorage on application startup. |
|
150 |
|
151 RelStorage is supposed to provide better performances than ZEO, notably under high load. RelStorage can also get |
|
152 benefit from many extensions (clustering, fail-over, hot-standby...) provided by these databases. |
|
153 |
|
154 |
|
155 .. _Newt.DB: |
|
156 |
|
157 3. NewtDB server |
|
158 ++++++++++++++++ |
|
159 |
|
160 NewtDB (http://www.newtdb.org/en/latest) is another ZODB storage implementation. It's using RelStorage but is |
|
161 dedicated to PostgreSQL (>= 9.5). |
|
162 |
|
163 NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB |
|
164 column. The JSON data supplements the native data to support indexing, search, and access from non-Python application. |
|
165 Because the JSON format is lossy, compared to the native format, the native format is still used for loading |
|
166 objects from the database. For this reason, the JSON data are read-only. |
|
167 |
|
168 Newt adds a search API for searching the Postgres JSON data and returning persistent objects. It also provides a |
|
169 convenient API for raw data searches. |
|
170 |
|
171 Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available |
|
172 if you need to switch from a classic RelStorage database to a Newt database. |
|
173 |
|
174 |
|
175 .. _ZODB.migration: |
|
176 |
|
177 Migration |
|
178 ''''''''' |
|
179 |
|
180 After installation, you can switch from a given storage to another one with the help of the *zodbconvert* command |
|
181 line script provided by RelStorage. |
|
182 |
|
183 This Python script is using a configuration file containing directives of both source and target databases, which can |
|
184 be any storage described in the previous sections (or can even use the same storage). |
|
185 |
|
186 Here is a sample configuration file to convert a ZODB from a ZEO to RelStorage: |
|
187 |
|
188 |
|
189 .. code-block:: html |
|
190 |
|
191 %import relstorage |
|
192 |
|
193 <zeoclient source> |
|
194 server zeo-server.mydomain:8100 |
|
195 storage pyams |
|
196 blob-dir /var/local/env/zeo/var/zeoclient/blobs |
|
197 shared-blob-dir false |
|
198 </zeoclient> |
|
199 |
|
200 <relstorage destination> |
|
201 keep-history false |
|
202 blob-dir /var/local/env/pyams/var/relstorage/blobs |
|
203 shared-blob-dir false |
|
204 <postgresql> |
|
205 dsn host='postgresql-server.mydomain' dbname='pyams_rs' user='pyams' password='xxxxxxxx' |
|
206 </postgresql> |
|
207 </relstorage> |
|
208 |
|
209 |