src/build/html/zodb.html
changeset 10 1d12ff3f036a
parent 9 562bde22e104
child 11 ba0b83b57c0c
equal deleted inserted replaced
9:562bde22e104 10:1d12ff3f036a
     1 
       
     2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
       
     3   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
       
     4 
       
     5 <html xmlns="http://www.w3.org/1999/xhtml">
       
     6   <head>
       
     7     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
       
     8     <title>Creating ZODB &#8212; PyAMS User Guide 0.1.0 documentation</title>
       
     9     <link rel="stylesheet" href="_static/pyramid.css" type="text/css" />
       
    10     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
       
    11     <script type="text/javascript">
       
    12       var DOCUMENTATION_OPTIONS = {
       
    13         URL_ROOT:    './',
       
    14         VERSION:     '0.1.0',
       
    15         COLLAPSE_INDEX: false,
       
    16         FILE_SUFFIX: '.html',
       
    17         HAS_SOURCE:  true,
       
    18         SOURCELINK_SUFFIX: '.txt'
       
    19       };
       
    20     </script>
       
    21     <script type="text/javascript" src="_static/jquery.js"></script>
       
    22     <script type="text/javascript" src="_static/underscore.js"></script>
       
    23     <script type="text/javascript" src="_static/doctools.js"></script>
       
    24     <link rel="index" title="Index" href="genindex.html" />
       
    25     <link rel="search" title="Search" href="search.html" />
       
    26     <link rel="next" title="Installing PyAMS" href="install.html" />
       
    27     <link rel="prev" title="Welcome to PyAMS_utils’s documentation!" href="index.html" />
       
    28 <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Neuton&amp;subset=latin" type="text/css" media="screen" charset="utf-8" />
       
    29 <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Nobile:regular,italic,bold,bolditalic&amp;subset=latin" type="text/css" media="screen" charset="utf-8" />
       
    30 <!--[if lte IE 6]>
       
    31 <link rel="stylesheet" href="_static/ie6.css" type="text/css" media="screen" charset="utf-8" />
       
    32 <![endif]-->
       
    33 
       
    34   </head>
       
    35   <body>
       
    36 
       
    37     <div class="related" role="navigation" aria-label="related navigation">
       
    38       <h3>Navigation</h3>
       
    39       <ul>
       
    40         <li class="right" style="margin-right: 10px">
       
    41           <a href="genindex.html" title="General Index"
       
    42              accesskey="I">index</a></li>
       
    43         <li class="right" >
       
    44           <a href="py-modindex.html" title="Python Module Index"
       
    45              >modules</a> |</li>
       
    46         <li class="right" >
       
    47           <a href="install.html" title="Installing PyAMS"
       
    48              accesskey="N">next</a> |</li>
       
    49         <li class="right" >
       
    50           <a href="index.html" title="Welcome to PyAMS_utils’s documentation!"
       
    51              accesskey="P">previous</a> |</li>
       
    52         <li class="nav-item nav-item-0"><a href="index.html">PyAMS User Guide 0.1.0 documentation</a> &#187;</li> 
       
    53       </ul>
       
    54     </div>  
       
    55 
       
    56     <div class="document">
       
    57       <div class="documentwrapper">
       
    58         <div class="bodywrapper">
       
    59           <div class="body" role="main">
       
    60             
       
    61   <div class="section" id="creating-zodb">
       
    62 <span id="zodb"></span><h1>Creating ZODB<a class="headerlink" href="#creating-zodb" title="Permalink to this headline">¶</a></h1>
       
    63 <p>PyAMS primarily relies on a ZODB (Zope Objects DataBase) to store it’s configuration. Other packages may
       
    64 rely on other database(s), but <em>PyAMS_content</em> package also stores it’s contents in a ZODB.</p>
       
    65 <p>As some PyAMS packages start several processes (“synchronization” is done via <strong>ØMQ</strong>), concurrent accesses are
       
    66 required on the ZODB (even when you start your application in “single process” mode); several ZODB storages
       
    67 implementations providing a shared access are available: <a class="reference internal" href="#zeo">ZEO</a>, <a class="reference internal" href="#relstorage">RelStorage</a> and <a class="reference internal" href="#newt-db">Newt.DB</a>.</p>
       
    68 <div class="section" id="installing-a-zeo-server">
       
    69 <span id="zeo"></span><h2>Installing a ZEO server<a class="headerlink" href="#installing-a-zeo-server" title="Permalink to this headline">¶</a></h2>
       
    70 <p>ZEO (Zope Enterprise Objects) is the first available implementation available for concurrent access to a
       
    71 FileStorage, provided through the ZEO package.</p>
       
    72 <p>ZEO package documentation and complete configuration settings are available on PyPI.</p>
       
    73 <div class="section" id="creating-initial-buildout">
       
    74 <h3>Creating initial buildout<a class="headerlink" href="#creating-initial-buildout" title="Permalink to this headline">¶</a></h3>
       
    75 <p>PyAMS provides a ZEO server scaffold, called <em>zeo_server</em>, generated via a <em>cookiecutter</em> template.</p>
       
    76 <p>A simple option to create a ZEO server is to create a buildout environment including <em>ZEO</em> and <em>ZODB</em> packages:</p>
       
    77 <div class="highlight-bash"><div class="highlight"><pre><span></span><span class="c1"># mkdir /var/local/</span>
       
    78 <span class="c1"># pip3 install virtualenv</span>
       
    79 <span class="c1"># virtualenv --python=python3.5 env</span>
       
    80 <span class="c1"># cd env</span>
       
    81 <span class="c1"># . bin/activate</span>
       
    82 <span class="o">(</span>env<span class="o">)</span> <span class="c1"># pip3.5 install cookiecutter</span>
       
    83 <span class="o">(</span>env<span class="o">)</span> <span class="c1"># cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/zeo_server</span>
       
    84 </pre></div>
       
    85 </div>
       
    86 <p><em>CookieCutter</em> will ask you for a small set of input variables that you can change or not:</p>
       
    87 <ul class="simple">
       
    88 <li><strong>pyams_release</strong>: version of PyAMS configuration file to use. “latest” (default value) will point to last release;
       
    89 you can also choose to point to a given release (“0.1.4” for example)</li>
       
    90 <li><strong>project_name</strong>: current environment name in “human form”</li>
       
    91 <li><strong>project_slug</strong>: “technical” package name, based on project name</li>
       
    92 <li><strong>eggs_directory</strong>: relative or absolute path to directory containing downloaded eggs; this directory can be
       
    93 shared with other projects (“eggs” as default)</li>
       
    94 <li><strong>run_user</strong>: user name under which ZEO process will run (“zeoadm” as default)</li>
       
    95 <li><strong>run_group</strong>: group name under which ZEO process will run (“zeo” as default)</li>
       
    96 <li><strong>zeo_server_port</strong>: listening port of ZEO server (“8100” as default)</li>
       
    97 <li><strong>zeo_monitor_port</strong>: listening port of ZEO monitor (“8101” as default)</li>
       
    98 <li><strong>zeo_storage</strong>: name of first ZEO storage; default value is based on project name</li>
       
    99 <li><strong>use_zeo_auth</strong>: specify if ZEO authentication should be used</li>
       
   100 <li><strong>zeo_auth_user</strong>: name of ZEO authenticated user (if ZEO authentication is used)</li>
       
   101 <li><strong>zeo_auth_password</strong>: password of ZEO authenticated user (if ZEO authentication is used)</li>
       
   102 <li><strong>zeo_pack_report</strong>: email address to which pack reports should be sent</li>
       
   103 <li><strong>logs_directory</strong>: absolute path to directory containing ZEO’s log files.</li>
       
   104 </ul>
       
   105 <p>A message is displayed after initialization to finalize environment creation:</p>
       
   106 <div class="code highlight-default"><div class="highlight"><pre><span></span><span class="n">Your</span> <span class="n">ZEO</span> <span class="n">environment</span> <span class="ow">is</span> <span class="n">initialized</span><span class="o">.</span>
       
   107 <span class="n">To</span> <span class="n">finalize</span> <span class="n">it</span><span class="s1">&#39;&#39;</span><span class="n">s</span> <span class="n">creation</span><span class="p">,</span> <span class="n">just</span> <span class="nb">type</span><span class="p">:</span>
       
   108 <span class="o">-</span> <span class="n">cd</span> <span class="n">zeo_server</span>
       
   109 <span class="o">-</span> <span class="n">python3</span><span class="o">.</span><span class="mi">5</span> <span class="n">bootstrap</span><span class="o">.</span><span class="n">py</span>
       
   110 <span class="o">-</span> <span class="o">./</span><span class="nb">bin</span><span class="o">/</span><span class="n">buildout</span>
       
   111 
       
   112 <span class="n">To</span> <span class="n">initialize</span> <span class="n">authentication</span> <span class="n">database</span><span class="p">,</span> <span class="n">please</span> <span class="n">run</span> <span class="n">following</span> <span class="n">command</span> <span class="n">after</span> <span class="n">buildout</span><span class="p">:</span>
       
   113 <span class="o">./</span><span class="nb">bin</span><span class="o">/</span><span class="n">zeopasswd</span> <span class="o">-</span><span class="n">f</span> <span class="n">etc</span><span class="o">/</span><span class="n">auth</span><span class="o">.</span><span class="n">db</span> <span class="o">-</span><span class="n">p</span> <span class="n">digest</span> <span class="o">-</span><span class="n">r</span> <span class="s2">&quot;ZEO_server&quot;</span> <span class="n">zeouser</span> <span class="n">xxxx</span>
       
   114 </pre></div>
       
   115 </div>
       
   116 </div>
       
   117 <div class="section" id="zeo-server-configuration">
       
   118 <h3>ZEO server configuration<a class="headerlink" href="#zeo-server-configuration" title="Permalink to this headline">¶</a></h3>
       
   119 <p>All ZEO configuration files are generated in “etc” subdirectory. These includes:</p>
       
   120 <ul class="simple">
       
   121 <li><strong>etc/zeo_server-zdaemon.conf</strong>: ZDaemon configuration file</li>
       
   122 <li><strong>etc/zeo_server-zeo.conf</strong>: ZEO server configuration file</li>
       
   123 <li><strong>etc/auth.db</strong>: ZEO authentication file; WARNING: this file is not created automatically, you have to create it
       
   124 after buildout.</li>
       
   125 </ul>
       
   126 <p>In these file names, always replace “zeo_server” with the value which was given to “project_slug” variable during
       
   127 <em>CookieCutter</em> template creation.</p>
       
   128 </div>
       
   129 <div class="section" id="zeo-server-tools">
       
   130 <h3>ZEO server tools<a class="headerlink" href="#zeo-server-tools" title="Permalink to this headline">¶</a></h3>
       
   131 <p>A set of system configuration files are produced to handle your ZEO environment. These includes:</p>
       
   132 <ul class="simple">
       
   133 <li><strong>etc/init.d/zeo-zeo_server</strong>: ZEO server start/stop script in Init-D format. Create a link to this file in
       
   134 <em>/etc/init.d</em> and update Init.d scripts (<em>update-rc.d zeo-zeo_server defaults</em>) to include ZEO in server start/stop
       
   135 process. You can also use this script to start/stop ZEO by hand with <em>start</em> and <em>stop</em> arguments.</li>
       
   136 <li><strong>etc/systemd/zeo-zeo_server.service</strong>: SystemD service configuration file for ZEO server. Create a link to this
       
   137 file in <em>/etc/systemd/system</em> and reload SystemD daemon (<em>systemctl daemon-reload</em>) before activating ZEO service
       
   138 (<em>systemctl enable zeo-zeo_server.service</em> and <em>systemctl start zeo-zeo_server.service</em>).</li>
       
   139 <li><strong>etc/logrotate.d/zeo-zeo_server</strong>: LogRotate configuration file for ZEO log files. Create a link to this file in
       
   140 <em>/etc/logrotate.d</em> to activate log rotation for ZEO server.</li>
       
   141 <li><strong>etc/cron.d/pack-zeo-zeo_server</strong>: Cron configuration file for ZEO database packing. Just create a link to this
       
   142 file in <em>/etc/cron.d</em> directory to enable ZODB packing on a weekly basis (by default).</li>
       
   143 </ul>
       
   144 <p>In these file names, always replace “zeo_server” with the value which was given to “project_slug” variable during
       
   145 <em>CookieCutter</em> template creation. All directory names are those used on a Debian GNU/Linux distribution and may have
       
   146 to be changed on other distributions.</p>
       
   147 </div>
       
   148 </div>
       
   149 <div class="section" id="installing-a-relstorage-server">
       
   150 <span id="relstorage"></span><h2>Installing a RelStorage server<a class="headerlink" href="#installing-a-relstorage-server" title="Permalink to this headline">¶</a></h2>
       
   151 <p>RelStorage (<a class="reference external" href="http://relstorage.readthedocs.io/en/latest">http://relstorage.readthedocs.io/en/latest</a>) is an alternate ZODB storage implementation, that stores
       
   152 Python pickles in a relational database; PostgreSQL (&gt;= 9.0), MySQL (&gt;= 5.0.32) and Oracle (&gt; 10g) databases are
       
   153 supported.</p>
       
   154 <p>To create a database compatible with RelStorage, you just have to install the database server and create a database
       
   155 dedicated to RelStorage; schema initialization is then completely done by RelStorage on application startup.</p>
       
   156 <p>RelStorage is supposed to provide better performances than ZEO, notably under high load. RelStorage can also get
       
   157 benefit from many extensions (clustering, fail-over, hot-standby…) provided by these databases.</p>
       
   158 </div>
       
   159 <div class="section" id="installing-a-newtdb-server">
       
   160 <span id="newt-db"></span><h2>Installing a NewtDB server<a class="headerlink" href="#installing-a-newtdb-server" title="Permalink to this headline">¶</a></h2>
       
   161 <p>NewtDB (<a class="reference external" href="http://www.newtdb.org/en/latest">http://www.newtdb.org/en/latest</a>) is another ZODB storage implementation. It’s using RelStorage but is
       
   162 dedicated to PostgreSQL (&gt;= 9.5).</p>
       
   163 <p>NewtDB adds conversion of data from the native serialization used by ZODB to JSON, stored in a PostgreSQL JSONB
       
   164 column. The JSON data supplements the native data to support indexing, search, and access from non-Python application.
       
   165 Because the JSON format is lossy, compared to the native format, the native format is still used for loading
       
   166 objects from the database. For this reason, the JSON data are read-only.</p>
       
   167 <p>Newt adds a search API for searching the Postgres JSON data and returning persistent objects. It also provides a
       
   168 convenient API for raw data searches.</p>
       
   169 <p>Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available
       
   170 if you need to switch from a classic RelStorage database to a Newt database.</p>
       
   171 </div>
       
   172 </div>
       
   173 
       
   174 
       
   175           </div>
       
   176         </div>
       
   177       </div>
       
   178       <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
       
   179         <div class="sphinxsidebarwrapper">
       
   180   <h3><a href="index.html">Table Of Contents</a></h3>
       
   181   <ul>
       
   182 <li><a class="reference internal" href="#">Creating ZODB</a><ul>
       
   183 <li><a class="reference internal" href="#installing-a-zeo-server">Installing a ZEO server</a><ul>
       
   184 <li><a class="reference internal" href="#creating-initial-buildout">Creating initial buildout</a></li>
       
   185 <li><a class="reference internal" href="#zeo-server-configuration">ZEO server configuration</a></li>
       
   186 <li><a class="reference internal" href="#zeo-server-tools">ZEO server tools</a></li>
       
   187 </ul>
       
   188 </li>
       
   189 <li><a class="reference internal" href="#installing-a-relstorage-server">Installing a RelStorage server</a></li>
       
   190 <li><a class="reference internal" href="#installing-a-newtdb-server">Installing a NewtDB server</a></li>
       
   191 </ul>
       
   192 </li>
       
   193 </ul>
       
   194 
       
   195   <h4>Previous topic</h4>
       
   196   <p class="topless"><a href="index.html"
       
   197                         title="previous chapter">Welcome to PyAMS_utils’s documentation!</a></p>
       
   198   <h4>Next topic</h4>
       
   199   <p class="topless"><a href="install.html"
       
   200                         title="next chapter">Installing PyAMS</a></p>
       
   201 <div id="searchbox" style="display: none" role="search">
       
   202   <h3>Quick search</h3>
       
   203     <form class="search" action="search.html" method="get">
       
   204       <div><input type="text" name="q" /></div>
       
   205       <div><input type="submit" value="Go" /></div>
       
   206       <input type="hidden" name="check_keywords" value="yes" />
       
   207       <input type="hidden" name="area" value="default" />
       
   208     </form>
       
   209 </div>
       
   210 <script type="text/javascript">$('#searchbox').show(0);</script>
       
   211         </div>
       
   212       </div>
       
   213       <div class="clearer"></div>
       
   214     </div>
       
   215     <div class="related" role="navigation" aria-label="related navigation">
       
   216       <h3>Navigation</h3>
       
   217       <ul>
       
   218         <li class="right" style="margin-right: 10px">
       
   219           <a href="genindex.html" title="General Index"
       
   220              >index</a></li>
       
   221         <li class="right" >
       
   222           <a href="py-modindex.html" title="Python Module Index"
       
   223              >modules</a> |</li>
       
   224         <li class="right" >
       
   225           <a href="install.html" title="Installing PyAMS"
       
   226              >next</a> |</li>
       
   227         <li class="right" >
       
   228           <a href="index.html" title="Welcome to PyAMS_utils’s documentation!"
       
   229              >previous</a> |</li>
       
   230         <li class="nav-item nav-item-0"><a href="index.html">PyAMS User Guide 0.1.0 documentation</a> &#187;</li> 
       
   231       </ul>
       
   232     </div>
       
   233     <div class="footer" role="contentinfo">
       
   234         &#169; Copyright 2017, Thierry Florac.
       
   235       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.5.
       
   236     </div>
       
   237   </body>
       
   238 </html>