|
1 .. _zodb: |
|
2 |
|
3 Creating ZODB |
|
4 ============= |
|
5 |
|
6 PyAMS primarily relies on a ZODB (Zope Objects DataBase) to store it's configuration. Other packages may |
|
7 rely on other database(s), but *PyAMS_content* package also stores it's contents in a ZODB. |
|
8 |
|
9 As some PyAMS packages start several processes ("synchronization" is done via **ØMQ**), concurrent accesses are |
|
10 required on the ZODB (even when you start your application in "single process" mode); several ZODB storages |
|
11 implementations providing a shared access are available: `ZEO`_, `RelStorage`_ and `Newt.DB`_. |
|
12 |
|
13 |
|
14 .. _ZEO: |
|
15 |
|
16 Installing a ZEO server |
|
17 +++++++++++++++++++++++ |
|
18 |
|
19 ZEO (Zope Enterprise Objects) is the first available implementation available for concurrent access to a |
|
20 FileStorage, provided through the ZEO package. |
|
21 |
|
22 ZEO package documentation and complete configuration settings are available on PyPI. |
|
23 |
|
24 Creating initial buildout |
|
25 ------------------------- |
|
26 |
|
27 PyAMS provides a ZEO server scaffold, called *zeo_server*, generated via a *cookiecutter* template. |
|
28 |
|
29 A simple option to create a ZEO server is to create a buildout environment including *ZEO* and *ZODB* packages: |
|
30 |
|
31 .. code-block:: bash |
|
32 |
|
33 # mkdir /var/local/ |
|
34 # pip3 install virtualenv |
|
35 # virtualenv --python=python3.5 env |
|
36 # cd env |
|
37 # . bin/activate |
|
38 (env) # pip3.5 install cookiecutter |
|
39 (env) # cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server |
|
40 |
|
41 *CookieCutter* will ask you for a small set of input variables that you can change or not: |
|
42 |
|
43 - **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release; |
|
44 you can also choose to point to a given release ("0.1.4" for example) |
|
45 |
|
46 - **project_name**: current environment name in "human form" |
|
47 |
|
48 - **project_slug**: "technical" package name, based on project name |
|
49 |
|
50 - **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be |
|
51 shared with other projects ("eggs" as default) |
|
52 |
|
53 - **run_user**: user name under which ZEO process will run ("zeoadm" as default) |
|
54 |
|
55 - **run_group**: group name under which ZEO process will run ("zeo" as default) |
|
56 |
|
57 - **zeo_server_port**: listening port of ZEO server ("8100" as default) |
|
58 |
|
59 - **zeo_monitor_port**: listening port of ZEO monitor ("8101" as default) |
|
60 |
|
61 - **zeo_storage**: name of first ZEO storage; default value is based on project name |
|
62 |
|
63 - **use_zeo_auth**: specify if ZEO authentication should be used |
|
64 |
|
65 - **zeo_auth_user**: name of ZEO authenticated user (if ZEO authentication is used) |
|
66 |
|
67 - **zeo_auth_password**: password of ZEO authenticated user (if ZEO authentication is used) |
|
68 |
|
69 - **zeo_pack_report**: email address to which pack reports should be sent |
|
70 |
|
71 - **logs_directory**: absolute path to directory containing ZEO's log files. |
|
72 |
|
73 A message is displayed after initialization to finalize environment creation: |
|
74 |
|
75 .. code:: |
|
76 |
|
77 Your ZEO environment is initialized. |
|
78 To finalize it''s creation, just type: |
|
79 - cd zeo_server |
|
80 - python3.5 bootstrap.py |
|
81 - ./bin/buildout |
|
82 |
|
83 To initialize authentication database, please run following command after buildout: |
|
84 ./bin/zeopasswd -f etc/auth.db -p digest -r "ZEO_server" zeouser xxxx |
|
85 |
|
86 ZEO server configuration |
|
87 ------------------------ |
|
88 |
|
89 All ZEO configuration files are generated in "etc" subdirectory. These includes: |
|
90 |
|
91 - **etc/zeo_server-zdaemon.conf**: ZDaemon configuration file |
|
92 |
|
93 - **etc/zeo_server-zeo.conf**: ZEO server configuration file |
|
94 |
|
95 - **etc/auth.db**: ZEO authentication file; WARNING: this file is not created automatically, you have to create it |
|
96 after buildout. |
|
97 |
|
98 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during |
|
99 *CookieCutter* template creation. |
|
100 |
|
101 ZEO server tools |
|
102 ---------------- |
|
103 |
|
104 A set of system configuration files are produced to handle your ZEO environment. These includes: |
|
105 |
|
106 - **etc/init.d/zeo-zeo_server**: ZEO server start/stop script in Init-D format. Create a link to this file in |
|
107 */etc/init.d* and update Init.d scripts (*update-rc.d zeo-zeo_server defaults*) to include ZEO in server start/stop |
|
108 process. You can also use this script to start/stop ZEO by hand with *start* and *stop* arguments. |
|
109 |
|
110 - **etc/systemd/zeo-zeo_server.service**: SystemD service configuration file for ZEO server. Create a link to this |
|
111 file in */etc/systemd/system* and reload SystemD daemon (*systemctl daemon-reload*) before activating ZEO service |
|
112 (*systemctl enable zeo-zeo_server.service* and *systemctl start zeo-zeo_server.service*). |
|
113 |
|
114 - **etc/logrotate.d/zeo-zeo_server**: LogRotate configuration file for ZEO log files. Create a link to this file in |
|
115 */etc/logrotate.d* to activate log rotation for ZEO server. |
|
116 |
|
117 - **etc/cron.d/pack-zeo-zeo_server**: Cron configuration file for ZEO database packing. Just create a link to this |
|
118 file in */etc/cron.d* directory to enable ZODB packing on a weekly basis (by default). |
|
119 |
|
120 In these file names, always replace "zeo_server" with the value which was given to "project_slug" variable during |
|
121 *CookieCutter* template creation. All directory names are those used on a Debian GNU/Linux distribution and may have |
|
122 to be changed on other distributions. |
|
123 |
|
124 |
|
125 .. _RelStorage: |
|
126 |
|
127 Installing a RelStorage server |
|
128 ++++++++++++++++++++++++++++++ |
|
129 |
|
130 RelStorage (http://relstorage.readthedocs.io/en/latest) is an alternate ZODB storage implementation, that stores |
|
131 Python pickles in a relational database; PostgreSQL (>= 9.0), MySQL (>= 5.0.32) and Oracle (> 10g) databases are |
|
132 supported. |
|
133 |
|
134 To create a database compatible with RelStorage, you just have to install the database server and create a database |
|
135 dedicated to RelStorage; schema initialization is then completely done by RelStorage on application startup. |
|
136 |
|
137 RelStorage is supposed to provide better performances than ZEO, notably under high load. RelStorage can also get |
|
138 benefit from many extensions (clustering, fail-over, hot-standby...) provided by these databases. |
|
139 |
|
140 |
|
141 .. _Newt.DB: |
|
142 |
|
143 Installing a NewtDB server |
|
144 ++++++++++++++++++++++++++ |
|
145 |
|
146 NewtDB (http://www.newtdb.org/en/latest) is another ZODB storage implementation. It's using RelStorage but is |
|
147 dedicated to PostgreSQL (>= 9.5). |
|
148 |
|
149 NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB |
|
150 column. The JSON data supplements the native data to support indexing, search, and access from non-Python application. |
|
151 Because the JSON format is lossy, compared to the native format, the native format is still used for loading |
|
152 objects from the database. For this reason, the JSON data are read-only. |
|
153 |
|
154 Newt adds a search API for searching the Postgres JSON data and returning persistent objects. It also provides a |
|
155 convenient API for raw data searches. |
|
156 |
|
157 Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available |
|
158 if you need to switch from a classic RelStorage database to a Newt database. |