diff -r eeef4052fcd1 -r f234f01d54f9 src/source/appinstall.rst --- a/src/source/appinstall.rst Thu Mar 29 13:42:57 2018 +0200 +++ b/src/source/appinstall.rst Tue Apr 03 18:17:48 2018 +0200 @@ -3,31 +3,39 @@ Installing PyAMS application ============================ -PyAMS default installation is based on `Buildout `_ utility. It's not mandatory to use a -virtual environment, but it allows you to have a better control over your Python resources. +**PyAMS** default installation is based on `Buildout `_ utility. + +.. tip:: + + Take a look at virtualenv_. This tool provide isolated Python environments, which are more practical than installing packages systemwide. + They also allow installing packages without administrator privileges. -Current PyAMS version is based and validated for Python 3.5; your Python environment must also include a C -compiler as well as development headers for Python, *libjpeg*, *libpng*, *libfreetype*, *libxml2*, *libxslt* and -eventually *libldap*, *libpq*, *libffi*, *libgdal* or *libzmq*. *Cython* compiler can also be useful to optimize +.. _virtualenv: https://virtualenv.pypa.io/en/stable/ + + +Current PyAMS version is based and validated for **Python 3.5**; your Python environment must also include a **C compiler** +as well as development headers for Python, *libjpeg*, *libpng*, *libfreetype*, *libxml2*, *libxslt* and +eventually *libldap*, *libpq*, *libffi*, *libgdal* or *libzmq*. **Cython** compiler can also be useful to optimize several packages. -PyAMS default components configuration also pre-suppose that the following external tools are available: -- a *Memcached* or *Redis* server, to store sessions and cache (can be changed through Beaker configuration) +.. note:: -Optional tools also include: + PyAMS default components configuration also pre-suppose that the following external services are available. -- an *LDAP* server for authentication + * Mandatory services: + + - a **ZODB shared server** (see :ref:`zodb` before). Several background processes needing a concurrent access to ZODB are started by PyAMS main process. Three ZODB storages are already provided through PyAMS: ZEO, RelStorage or Newt.db. -- an *ElasticSearch* server for full text indexing (see *PyAMS_content_es* package) + - a **Memcached** or **Redis*** server, to store sessions and cache -- a *WebSockets* server using AsyncIO. This is used to manage notifications (see *PyAMS_notify* and *PyAMS_notify_ws* - packages). An *out of the box* environment can be built using *pyams_notify* scaffold. + * Optional services: + - an **LDAP server** for authentication -PyAMS also needs that you use a ZODB shared server, as several background processes needing a concurrent access to ZODB -are started by PyAMS main process. Three ZODB storages are already provided through PyAMS: ZEO, RelStorage or Newt.db. -See :ref:`zodb` to know how to initialize database with the help of PyAMS tools. + - an **ElasticSearch server** for full text indexing (see *PyAMS_content_es* package) + + - a **WebSockets server** using AsyncIO. This is used to manage notifications (see *PyAMS_notify* and *PyAMS_notify_ws* packages). An *out of the box* environment can be built using *pyams_notify* scaffold. Creating initial buildout @@ -39,50 +47,50 @@ .. code-block:: bash - # mkdir /var/local/ - # pip3 install virtualenv - # virtualenv --python=python3.5 env - # cd env - # . bin/activate - (env) # pip3.5 install cookiecutter - (env) # cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/pyams + $ mkdir /var/local/ + $ pip3 install virtualenv + $ virtualenv --python=python3.5 env + $ cd env + $ . bin/activate + (env) $ pip3.5 install cookiecutter + (env) $ cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/pyams -*CookieCutter* will ask you for a small set of input variables that you can change or not: +*CookieCutter* will ask you for a small set of input variables that you can change or not (**parameter** [*default value*]:) -- **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release; +- **pyams_release** [*latest*] : version of PyAMS configuration file to use. "latest" will point to last release; you can also choose to point to a given release ("0.1.4" for example) -- **project_name**: current environment name in "human form" +- **project_name** [*PyAMS*]: current environment name in "human form" -- **project_slug**: "technical" package name, based on project name +- **project_slug**: [*pyams*] "technical" package name, based on project name -- **virtual_hostname**: Apache virtual-host name +- **virtual_hostname** [*pyams.mydomain.com*]: Apache virtual-host name -- **webapp_name**: web application package name ("webapp" as default) +- **webapp_name** [*webapps*]: web application package name -- **webapp_port**: TCP/IP port to use when running application outside Apache ("6543" as default) +- **webapp_port** [*6543*]: TCP/IP port to use when running application outside Apache. -- **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be - shared with other projects ("eggs" as default) +- **eggs_directory** [*eggs*]: relative or absolute path to directory containing downloaded eggs; this directory can be + shared with other projects. -- **logs_directory**: absolute path to directory containing Apache's log files +- **logs_directory** [*/var/log*]: absolute path to directory containing Apache's log files -- **run_user**: user name under which Apache process will run ("www-data" as default) +- **run_user** [*www-data*]: user name under which Apache process will run -- **run_group**: group name under which Apache process will run ("www-data" as default) +- **run_group** [*www-data*]: group name under which Apache process will run -- **beaker_backend**: name of Beaker backend to use to store sessions and cache data ("redis" as default) +- **beaker_backend** : name of Beaker backend to use to store sessions and cache data ("redis" as default) -- **beaker_server**: IP address and port of Beaker backend server ("127.0.0.1:6379" as default) +- **beaker_server** [*127.0.0.1:6379*]: IP address and port of Beaker backend server -- **db_type**: ZODB database storage; available options include ZEO, RelStorage and NewtDB +- **db_type** [*zeo*]: ZODB database storage; available options include ZEO, RelStorage and NewtDB -- **db_host**: IP address of database server ("127.0.0.1" as default); WARNING: database server installation +- **db_host** [*127.0.0.1*]: IP address of database server ("127.0.0.1" as default); WARNING: database server installation is not part of application installation; another "zeo_server" cookiecutter recipe is available for ZEO -- **db_port**: listening port of database server ("8100" is given as default for ZEO) +- **db_port** [*8100*]: listening port of database server ("8100" is given as default for ZEO) -- **db_name**: database or ZEO storage name to use +- **db_name** [*pyams*]: database or ZEO storage name to use - **db_username**: database user name @@ -90,7 +98,7 @@ - **zeo_realm**: ZEO authentication realm -- **blobs_dir**: local directory to use to store cache of ZODB blobs; cache size is limited to 10GB as default +- **blobs_dir** [*/var/db/blobs*]: local directory to use to store cache of ZODB blobs; cache size is limited to 10GB as default - **use_postgresql**: specify if PostgreSQL access is required; if so, please check that PostgreSQL development files are available to compile PsycoPG2 extension @@ -115,27 +123,23 @@ - **smtp_server_name**: "human" name given to SMTP server ("pyams" as default) -- **pyams_scheduler**: TCP/IP address and port to use to access PyAMS tasks scheduler process ("127.0.0.1:5555" as - default); see :ref:`pyams_scheduler` +- **pyams_scheduler** [*127.0.0.1:5555*]: TCP/IP address and port to use to access PyAMS tasks scheduler process (see :ref:`pyams_scheduler`) -- **start_scheduler**: boolean value to indicate if scheduler process is started by this application instance +- **start_scheduler** [*True*]: boolean value to indicate if scheduler process is started by this application instance -- **pyams_medias_converter**: TCP/IP address and port to use to access PyAMS medias converter process ("127.0.0.1:5556" - as default); see :ref:`pyams_medias` +- **pyams_medias_converter** [*127.0.0.1:5556*]: TCP/IP address and port to use to access PyAMS medias converter process (see :ref:`pyams_media`) -- **start_medias_converter**: boolean value to indicate if medias converter process is started by this application +- **start_medias_converter** [*True*]: boolean value to indicate if medias converter process is started by this application instance -- **pyams_es_indexer**: TCP/IP address and port to use to access PyAMS Elasticsearch indexer process ("127.0.0.1:5557" - as default); see :ref:`pyams_content_es` +- **pyams_es_indexer** [*127.0.0.1:5557*] : TCP/IP address and port to use to access PyAMS Elasticsearch indexer process (see :ref:`pyams_content_es`) - **start_es_indexer** boolean value to indicate if Elasticsearch indexer process is started by this application instance - **use_notifications**: specify if PyAMS notifications services are to be used (see :ref:`pyams_notify`) -- **pyams_ws_notify**: TCP/IP address and port of PyAMS websockets server managing notifications service - ("127.0.0.1:8081" as default) +- **pyams_ws_notify** [*127.0.0.1:8081*]: TCP/IP address and port of PyAMS websockets server managing notifications service - **lexicon_languages**: NLTK lexicon languages to use ("en:english fr:french" as default) @@ -145,17 +149,13 @@ that *libgdal* development files are available; on Debian (and maybe others), you have to specify environment variables (see below). +.. tip:: -Boolean values accept "true", "yes", "on" or "1" (in any case) as *true* values, and anything else as *false*; setting -any option to empty string will keep the default proposed value. + 1. Setting any option to empty string will keep the default proposed value. + 2. Boolean values accept "true", "yes", "on" or "1" (in any case) as *true* values, and anything else as *false*; You can then check, and eventually update, the proposed Buildout configuration file *buildout.cfg*, to add or remove -packages or update settings to your needs. Then finalize Bootstrap initialization: - -.. code-block:: bash - - (env) # python3.5 bootstrap.py - (env) # ./bin/buildout +packages or update settings to your needs. This last operation can be quite long, as many packages have to downloaded, compiled and installed in the virtual environment. If you encounter any compile error, just install the required dependencies and restart the buildout. @@ -166,22 +166,32 @@ .. code-block:: bash - (env) # export C_INCLUDE_PATH=/usr/include/gdal - (env) # export CPLUS_INCLUDE_PATH=/usr/include/gdal + (env) $ export C_INCLUDE_PATH=/usr/include/gdal + (env) $ export CPLUS_INCLUDE_PATH=/usr/include/gdal -**WARNING**: you have to check also that your *libgdal* release is matching "GDAL" release given in PyAMS + +.. WARNING:: + You have to check also that your *libgdal* release is matching "GDAL" release given in PyAMS configuration file (actually 2.1.0). - for *cx_Oracle*, which is required if you use Oracle database connections, use: .. code-block:: bash - (env) # export ORACLE_HOME=/usr/lib/oracle/12.1/client64 + (env) $ export ORACLE_HOME=/usr/lib/oracle/12.1/client64 These examples are given for Debian GNU/Linux. You may have to adapt configuration based on your own Linux distribution and packages versions. +**Then finalize Bootstrap initialization:** + + +.. code-block:: bash + + (env) $ python3.5 bootstrap.py + (env) $ ./bin/buildout + Environment settings -------------------- @@ -207,24 +217,24 @@ update them carefully before initializing your application database! -Initializing the database -------------------------- +Initializing the ZODB database +========================= -When you have downloaded and installed all required packages, you have to initialize the database so that all +After, you have downloaded and installed all required packages, you have to initialize the database so that all required components are available. From a shell, just type: .. code-block:: bash - (env) # ./bin/pyams_upgrade etc/development.ini + (env) $ ./bin/pyams_upgrade etc/development.ini This process requires that every package is correctly included into *pyramid.includes* directive from selected configuration file. Initializing Elasticsearch index --------------------------------- +================================ If you want to use an Elasticsearch index, you have to initialize index settings and mappings; the Ingest attachment plug-in is also required to handle attachments correctly. @@ -234,26 +244,26 @@ .. code-block:: bash - (env) # cd /var/local/src/pyams/pyams_content_es - (env) # curl --noproxy localhost -XDELETE http://localhost:9200/pyams (1) - (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams -d @index-settings.json + (env) $ cd /var/local/src/pyams/pyams_content_es + (env) $ curl --noproxy localhost -XDELETE http://localhost:9200/pyams (1) + (env) $ curl --noproxy localhost -XPUT http://localhost:9200/pyams -d @index-settings.json - (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfNewsEvent/_mapping -d @mappings/WfNewsEvent.json - (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfTopic/_mapping -d @mappings/WfTopic.json - (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfBlogPost/_mapping -d @mappings/WfBlogPost.json + (env) $ curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfNewsEvent/_mapping -d @mappings/WfNewsEvent.json + (env) $ curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfTopic/_mapping -d @mappings/WfTopic.json + (env) $ curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfBlogPost/_mapping -d @mappings/WfBlogPost.json (1) If 'pyams' is defined as Elasticsearch index name. -NLTK initialization -------------------- +Initializing NLTK +================= Some NLTK (Natural Language Toolkit) tokenizers and stopwords utilities are used to index fulltext contents elements. This package requires downloading and configuration of several elements which are done as follow: .. code-block:: bash - (end) # ./bin/py + (end) $ ./bin/py >>> import nltk >>> nltk.download() NLTK Downloader @@ -299,14 +309,14 @@ Starting the application ------------------------- +======================== When database upgrade process has ended, you can start the web application process with the standard Pyramid's *pserve* command line tool: .. code-block:: bash - (env) # ./bin/pserve etc/development.ini + (env) $ ./bin/pserve etc/development.ini In standard debug mode, all registered components are displayed in the console, until the final line (here using ZEO): @@ -379,3 +389,4 @@ From this point, you can launch a browser and open URL *http://127.0.0.1:6543/admin* to get access to PyAMS management interface; default login is "admin/admin", that you may change as soon as possible (see :ref:`pyams_security`)!!. +