src/source/appinstall.rst
changeset 99 b2be9a32f3fc
child 104 942151432421
equal deleted inserted replaced
-1:000000000000 99:b2be9a32f3fc
       
     1 .. _appinstall:
       
     2 
       
     3 .. _install:
       
     4 
       
     5 Installing PyAMS application
       
     6 ============================
       
     7 
       
     8 **PyAMS** default installation is based on `Buildout <http://www.buildout.org>`_ utility.
       
     9 
       
    10 .. tip::
       
    11 
       
    12     Take a look at virtualenv_. This tool provide isolated Python environments, which are more practical than installing packages systemwide.
       
    13     They also allow installing packages without administrator privileges.
       
    14 
       
    15 .. _virtualenv: https://virtualenv.pypa.io/en/stable/
       
    16 
       
    17 
       
    18 Current PyAMS version is based and validated for **Python 3.5**; your Python environment must also include a **C compiler**
       
    19 as well as development headers for Python, *libjpeg*, *libpng*, *libfreetype*, *libxml2*, *libxslt* and
       
    20 eventually *libldap*, *libpq*, *libffi*, *libgdal* or *libzmq*. **Cython** compiler can also be useful to optimize
       
    21 several packages.
       
    22 
       
    23 
       
    24 .. note::
       
    25 
       
    26     PyAMS default components configuration also pre-suppose that the following external services are available.
       
    27 
       
    28     * Mandatory services:
       
    29 
       
    30        - a **ZODB shared server** (see :ref:`zodb` before). Several background processes needing a concurrent access to ZODB are started by PyAMS main process. Three ZODB storages are already provided through PyAMS: ZEO, RelStorage or Newt.db.
       
    31 
       
    32        - a **Memcached** or **Redis*** server, to store sessions and cache
       
    33 
       
    34     * Optional services:
       
    35 
       
    36        - an **LDAP server** for authentication
       
    37 
       
    38        - an **ElasticSearch server** for full text indexing (see *PyAMS_content_es* package)
       
    39 
       
    40        - a **WebSockets server** using AsyncIO. This is used to manage notifications (see *PyAMS_notify* and *PyAMS_notify_ws* packages). An *out of the box* environment can be built using *pyams_notify* scaffold.
       
    41 
       
    42 
       
    43 Creating initial buildout
       
    44 -------------------------
       
    45 
       
    46 PyAMS provides a new Pyramid scaffold, called *pyams*, generated via a *cookiecutter* template.
       
    47 
       
    48 A simple option to install PyAMS is to create a buildout environment including *Pyramid* and all *PyAMS* packages:
       
    49 
       
    50 .. code-block:: bash
       
    51 
       
    52     $ mkdir /var/local/
       
    53     $ pip3 install virtualenv
       
    54     $ virtualenv --python=python3.5 env
       
    55     $ cd env
       
    56     $ . bin/activate
       
    57     (env) $ pip3.5 install cookiecutter
       
    58     (env) $ cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/pyams
       
    59 
       
    60 *CookieCutter* will ask you for a small set of input variables that you can change or not (**parameter** [*default value*]:)
       
    61 
       
    62 - **pyams_release** [*latest*] : version of PyAMS configuration file to use. "latest" will point to last release;
       
    63   you can also choose to point to a given release ("0.1.4" for example)
       
    64 
       
    65 - **project_name** [*PyAMS*]: current environment name in "human form"
       
    66 
       
    67 - **project_slug**: [*pyams*] "technical" package name, based on project name
       
    68 
       
    69 - **virtual_hostname** [*pyams.mydomain.com*]: Apache virtual-host name
       
    70 
       
    71 - **webapp_name** [*webapps*]: web application package name
       
    72 
       
    73 - **webapp_port** [*6543*]: TCP/IP port to use when running application outside Apache.
       
    74 
       
    75 - **eggs_directory** [*eggs*]: relative or absolute path to directory containing downloaded eggs; this directory can be
       
    76   shared with other projects.
       
    77 
       
    78 - **logs_directory** [*/var/log*]: absolute path to directory containing Apache's log files
       
    79 
       
    80 - **run_user** [*www-data*]: user name under which Apache process will run
       
    81 
       
    82 - **run_group** [*www-data*]: group name under which Apache process will run
       
    83 
       
    84 - **beaker_backend** : name of Beaker backend to use to store sessions and cache data ("redis" as default)
       
    85 
       
    86 - **beaker_server** [*127.0.0.1:6379*]: IP address and port of Beaker backend server
       
    87 
       
    88 - **db_type** [*zeo*]: ZODB database storage; available options include ZEO, RelStorage and NewtDB
       
    89 
       
    90 - **db_host** [*127.0.0.1*]: IP address of database server ("127.0.0.1" as default); WARNING: database server installation
       
    91   is not part of application installation; another "zeo_server" cookiecutter recipe is available for ZEO
       
    92 
       
    93 - **db_port** [*8100*]: listening port of database server ("8100" is given as default for ZEO)
       
    94 
       
    95 - **db_name** [*pyams*]: database or ZEO storage name to use
       
    96 
       
    97 - **db_username**: database user name
       
    98 
       
    99 - **db_password**: database password
       
   100 
       
   101 - **zeo_realm**: ZEO authentication realm
       
   102 
       
   103 - **blobs_dir** [*/var/db/blobs*]: local directory to use to store cache of ZODB blobs; cache size is limited to 10GB as default
       
   104 
       
   105 - **use_postgresql**: specify if PostgreSQL access is required; if so, please check that PostgreSQL development files
       
   106   are available to compile PsycoPG2 extension
       
   107 
       
   108 - **use_oracle**: specify if Oracle access is required; if so, please check that Oracle development files are
       
   109   available to compile cx_Oracle extension, and that ORACLE_HOME environment variable is correctly defined (see below)
       
   110 
       
   111 - **use_ldap**: specify if LDAP access will be required for authentication
       
   112 
       
   113 - **use_elasticsearch**: specify if an ElasticSearch server will be used for indexation
       
   114 
       
   115 - **elasticsearch_server**: URL used to access Elasticsearch server ("http://127.0.0.1:9200" as default); this URL can
       
   116   include login and password ("http://login:password@127.0.0.1:9200"), if required...
       
   117 
       
   118 - **elasticsearch_index**: name of Elasticsearch index to use ("pyams" as default)
       
   119 
       
   120 - **create_elasticsearch_index**: specify if Elasticsearch index should be created after installation is complete
       
   121 
       
   122 - **define_elasticsearch_mappings** : specify if Elasticsearch mappings should be defined after installation is complete
       
   123 
       
   124 - **smtp_server**: DNS name of SMTP server ("localhost" as default)
       
   125 
       
   126 - **smtp_server_name**: "human" name given to SMTP server ("pyams" as default)
       
   127 
       
   128 - **pyams_scheduler** [*127.0.0.1:5555*]: TCP/IP address and port to use to access PyAMS tasks scheduler process (see :ref:`pyams_scheduler`)
       
   129 
       
   130 - **start_scheduler** [*True*]: boolean value to indicate if scheduler process is started by this application instance
       
   131 
       
   132 - **pyams_medias_converter** [*127.0.0.1:5556*]: TCP/IP address and port to use to access PyAMS medias converter process (see :ref:`pyams_media`)
       
   133 
       
   134 - **start_medias_converter** [*True*]: boolean value to indicate if medias converter process is started by this application
       
   135   instance
       
   136 
       
   137 - **pyams_es_indexer** [*127.0.0.1:5557*] : TCP/IP address and port to use to access PyAMS Elasticsearch indexer process  (see :ref:`pyams_content_es`)
       
   138 
       
   139 - **start_es_indexer** boolean value to indicate if Elasticsearch indexer process is started by this application
       
   140   instance
       
   141 
       
   142 - **use_notifications**: specify if PyAMS notifications services are to be used (see :ref:`pyams_notify`)
       
   143 
       
   144 - **pyams_ws_notify** [*127.0.0.1:8081*]: TCP/IP address and port of PyAMS websockets server managing notifications service
       
   145 
       
   146 - **lexicon_languages**: NLTK lexicon languages to use ("en:english fr:french" as default)
       
   147 
       
   148 - **extension_package**: name of a PyAMS extension package to include in environment configuration
       
   149 
       
   150 - **need_pyams_gis**: specify if PyAMS GIS features are to be used by given extension package; if so, please check
       
   151   that *libgdal* development files are available; on Debian (and maybe others), you have to specify environment
       
   152   variables (see below).
       
   153 
       
   154 .. tip::
       
   155 
       
   156     1. Setting any option to empty string will keep the default proposed value.
       
   157     2. Boolean values accept "true", "yes", "on" or "1" (in any case) as *true* values, and anything else as *false*;
       
   158 
       
   159 You can then check, and eventually update, the proposed Buildout configuration file *buildout.cfg*, to add or remove
       
   160 packages or update settings to your needs.
       
   161 
       
   162 This last operation can be quite long, as many packages have to downloaded, compiled and installed in the virtual
       
   163 environment. If you encounter any compile error, just install the required dependencies and restart the buildout.
       
   164 
       
   165 Some dependencies can require the definition of custom environment variables before running *buildout*, like:
       
   166 
       
   167 - for *libgdal*, which is required by **PyAMS_gis** package, use:
       
   168 
       
   169 .. code-block:: bash
       
   170 
       
   171     (env) $ export C_INCLUDE_PATH=/usr/include/gdal
       
   172     (env) $ export CPLUS_INCLUDE_PATH=/usr/include/gdal
       
   173 
       
   174 
       
   175 .. WARNING::
       
   176     You have to check also that your *libgdal* release is matching "GDAL" release given in PyAMS
       
   177     configuration file (actually 2.1.0).
       
   178 
       
   179 - for *cx_Oracle*, which is required if you use Oracle database connections, use:
       
   180 
       
   181 .. code-block:: bash
       
   182 
       
   183     (env) $ export ORACLE_HOME=/usr/lib/oracle/12.1/client64
       
   184 
       
   185 These examples are given for Debian GNU/Linux. You may have to adapt configuration based on your own Linux
       
   186 distribution and packages versions.
       
   187 
       
   188 
       
   189 **Then finalize Bootstrap initialization:**
       
   190 
       
   191 .. code-block:: bash
       
   192 
       
   193     (env) $ python3.5 bootstrap.py
       
   194     (env) $ ./bin/buildout
       
   195 
       
   196 
       
   197 .. seealso::
       
   198 
       
   199     `zc.buildout <https://pypi.python.org/pypi/zc.buildout>`_ is a powerful
       
   200     tool for creating repeatable builds of a given software configuration
       
   201     and environment.
       
   202 
       
   203 
       
   204 Environment settings
       
   205 --------------------
       
   206 
       
   207 The project generated from *pyams* scaffold is based on default Pyramid's *zodb* scaffold, but it adds:
       
   208 
       
   209 - a custom application factory, in the *webapp* directory (see :ref:`site`)
       
   210 
       
   211 - a set of directories to store runtime data, in the *var* directory; each directory contains a *README.txt* file
       
   212   which should be self-explanatory to indicate what this directory should contain, including a ZEO cache
       
   213 
       
   214 - a set of configuration files, in the *etc* directory; here are standard *development.ini* and *production.ini*
       
   215   configuration files, a ZODB configuration files (*zodb-zeo.conf*) for a ZEO client storage and two Apache
       
   216   configurations (for Apache 2.2 and 2.4) using *mod_wsgi*.
       
   217 
       
   218 Once the project have been created from the scaffold, you are free to update all the configuration files.
       
   219 
       
   220 If you need to add packages to the environment, you have to add them to the *buildout.cfg* file **AND** to the INI
       
   221 file (in the *pyramid.includes* section) before running the *buildout* another time; don't forget to add the
       
   222 requested version at the end of *buildout.cfg* file, as Buildout is not configured by default to automatically
       
   223 download the last release of a given unknown package.
       
   224 
       
   225 *development.ini* and *production.ini* files contain many commented directives related to PyAMS components. Read and
       
   226 update them carefully before initializing your application database!
       
   227 
       
   228 
       
   229 .. _zodbinit:
       
   230 
       
   231 Initializing the ZODB database
       
   232 ------------------------------
       
   233 
       
   234 After, you have downloaded and installed all required packages, you have to initialize the database so that all
       
   235 required components are available.
       
   236 
       
   237 From a shell, just type:
       
   238 
       
   239 .. code-block:: bash
       
   240 
       
   241     (env) $ ./bin/pyams_upgrade etc/development.ini
       
   242 
       
   243 This process requires that every package is correctly included into *pyramid.includes* directive from selected
       
   244 configuration file.