--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/src/source/appinstall.rst Tue Mar 06 11:06:45 2018 +0100
@@ -0,0 +1,381 @@
+.. _appinstall:
+
+Installing PyAMS application
+============================
+
+PyAMS default installation is based on `Buildout <http://www.buildout.org>`_ utility. It's not mandatory to use a
+virtual environment, but it allows you to have a better control over your Python resources.
+
+Current PyAMS version is based and validated for Python 3.5; your Python environment must also include a C
+compiler as well as development headers for Python, *libjpeg*, *libpng*, *libfreetype*, *libxml2*, *libxslt* and
+eventually *libldap*, *libpq*, *libffi*, *libgdal* or *libzmq*. *Cython* compiler can also be useful to optimize
+several packages.
+
+PyAMS default components configuration also pre-suppose that the following external tools are available:
+
+- a *Memcached* or *Redis* server, to store sessions and cache (can be changed through Beaker configuration)
+
+Optional tools also include:
+
+- an *LDAP* server for authentication
+
+- an *ElasticSearch* server for full text indexing (see *PyAMS_content_es* package)
+
+- a *WebSockets* server using AsyncIO. This is used to manage notifications (see *PyAMS_notify* and *PyAMS_notify_ws*
+ packages). An *out of the box* environment can be built using *pyams_notify* scaffold.
+
+
+PyAMS also needs that you use a ZODB shared server, as several background processes needing a concurrent access to ZODB
+are started by PyAMS main process. Three ZODB storages are already provided through PyAMS: ZEO, RelStorage or Newt.db.
+See :ref:`zodb` to know how to initialize database with the help of PyAMS tools.
+
+
+Creating initial buildout
+-------------------------
+
+PyAMS provides a new Pyramid scaffold, called *pyams*, generated via a *cookiecutter* template.
+
+A simple option to install PyAMS is to create a buildout environment including *Pyramid* and all *PyAMS* packages:
+
+.. code-block:: bash
+
+ # mkdir /var/local/
+ # pip3 install virtualenv
+ # virtualenv --python=python3.5 env
+ # cd env
+ # . bin/activate
+ (env) # pip3.5 install cookiecutter
+ (env) # cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/pyams
+
+*CookieCutter* will ask you for a small set of input variables that you can change or not:
+
+- **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release;
+ you can also choose to point to a given release ("0.1.4" for example)
+
+- **project_name**: current environment name in "human form"
+
+- **project_slug**: "technical" package name, based on project name
+
+- **virtual_hostname**: Apache virtual-host name
+
+- **webapp_name**: web application package name ("webapp" as default)
+
+- **webapp_port**: TCP/IP port to use when running application outside Apache ("6543" as default)
+
+- **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be
+ shared with other projects ("eggs" as default)
+
+- **logs_directory**: absolute path to directory containing Apache's log files
+
+- **run_user**: user name under which Apache process will run ("www-data" as default)
+
+- **run_group**: group name under which Apache process will run ("www-data" as default)
+
+- **beaker_backend**: name of Beaker backend to use to store sessions and cache data ("redis" as default)
+
+- **beaker_server**: IP address and port of Beaker backend server ("127.0.0.1:6379" as default)
+
+- **db_type**: ZODB database storage; available options include ZEO, RelStorage and NewtDB
+
+- **db_host**: IP address of database server ("127.0.0.1" as default); WARNING: database server installation
+ is not part of application installation; another "zeo_server" cookiecutter recipe is available for ZEO
+
+- **db_port**: listening port of database server ("8100" is given as default for ZEO)
+
+- **db_name**: database or ZEO storage name to use
+
+- **db_username**: database user name
+
+- **db_password**: database password
+
+- **zeo_realm**: ZEO authentication realm
+
+- **blobs_dir**: local directory to use to store cache of ZODB blobs; cache size is limited to 10GB as default
+
+- **use_postgresql**: specify if PostgreSQL access is required; if so, please check that PostgreSQL development files
+ are available to compile PsycoPG2 extension
+
+- **use_oracle**: specify if Oracle access is required; if so, please check that Oracle development files are
+ available to compile cx_Oracle extension, and that ORACLE_HOME environment variable is correctly defined (see below)
+
+- **use_ldap**: specify if LDAP access will be required for authentication
+
+- **use_elasticsearch**: specify if an ElasticSearch server will be used for indexation
+
+- **elasticsearch_server**: URL used to access Elasticsearch server ("http://127.0.0.1:9200" as default); this URL can
+ include login and password ("http://login:password@127.0.0.1:9200"), if required...
+
+- **elasticsearch_index**: name of Elasticsearch index to use ("pyams" as default)
+
+- **create_elasticsearch_index**: specify if Elasticsearch index should be created after installation is complete
+
+- **define_elasticsearch_mappings** : specify if Elasticsearch mappings should be defined after installation is complete
+
+- **smtp_server**: DNS name of SMTP server ("localhost" as default)
+
+- **smtp_server_name**: "human" name given to SMTP server ("pyams" as default)
+
+- **pyams_scheduler**: TCP/IP address and port to use to access PyAMS tasks scheduler process ("127.0.0.1:5555" as
+ default); see :ref:`pyams_scheduler`
+
+- **start_scheduler**: boolean value to indicate if scheduler process is started by this application instance
+
+- **pyams_medias_converter**: TCP/IP address and port to use to access PyAMS medias converter process ("127.0.0.1:5556"
+ as default); see :ref:`pyams_medias`
+
+- **start_medias_converter**: boolean value to indicate if medias converter process is started by this application
+ instance
+
+- **pyams_es_indexer**: TCP/IP address and port to use to access PyAMS Elasticsearch indexer process ("127.0.0.1:5557"
+ as default); see :ref:`pyams_content_es`
+
+- **start_es_indexer** boolean value to indicate if Elasticsearch indexer process is started by this application
+ instance
+
+- **use_notifications**: specify if PyAMS notifications services are to be used (see :ref:`pyams_notify`)
+
+- **pyams_ws_notify**: TCP/IP address and port of PyAMS websockets server managing notifications service
+ ("127.0.0.1:8081" as default)
+
+- **lexicon_languages**: NLTK lexicon languages to use ("en:english fr:french" as default)
+
+- **extension_package**: name of a PyAMS extension package to include in environment configuration
+
+- **need_pyams_gis**: specify if PyAMS GIS features are to be used by given extension package; if so, please check
+ that *libgdal* development files are available; on Debian (and maybe others), you have to specify environment
+ variables (see below).
+
+
+Boolean values accept "true", "yes", "on" or "1" (in any case) as *true* values, and anything else as *false*; setting
+any option to empty string will keep the default proposed value.
+
+You can then check, and eventually update, the proposed Buildout configuration file *buildout.cfg*, to add or remove
+packages or update settings to your needs. Then finalize Bootstrap initialization:
+
+.. code-block:: bash
+
+ (env) # python3.5 bootstrap.py
+ (env) # ./bin/buildout
+
+This last operation can be quite long, as many packages have to downloaded, compiled and installed in the virtual
+environment. If you encounter any compile error, just install the required dependencies and restart the buildout.
+
+Some dependencies can require the definition of custom environment variables before running *buildout*, like:
+
+- for *libgdal*, which is required by **PyAMS_gis** package, use:
+
+.. code-block:: bash
+
+ (env) # export C_INCLUDE_PATH=/usr/include/gdal
+ (env) # export CPLUS_INCLUDE_PATH=/usr/include/gdal
+
+**WARNING**: you have to check also that your *libgdal* release is matching "GDAL" release given in PyAMS
+configuration file (actually 2.1.0).
+
+- for *cx_Oracle*, which is required if you use Oracle database connections, use:
+
+.. code-block:: bash
+
+ (env) # export ORACLE_HOME=/usr/lib/oracle/12.1/client64
+
+These examples are given for Debian GNU/Linux. You may have to adapt configuration based on your own Linux
+distribution and packages versions.
+
+
+Environment settings
+--------------------
+
+The project generated from *pyams* scaffold is based on default Pyramid's *zodb* scaffold, but it adds:
+
+- a custom application factory, in the *webapp* directory (see :ref:`site`)
+
+- a set of directories to store runtime data, in the *var* directory; each directory contains a *README.txt* file
+ which should be self-explanatory to indicate what this directory should contain, including a ZEO cache
+
+- a set of configuration files, in the *etc* directory; here are standard *development.ini* and *production.ini*
+ configuration files, a ZODB configuration files (*zodb-zeo.conf*) for a ZEO client storage and two Apache
+ configurations (for Apache 2.2 and 2.4) using *mod_wsgi*.
+
+Once the project have been created from the scaffold, you are free to update all the configuration files.
+
+If you need to add packages to the environment, you have to add them to the *buildout.cfg* file **AND** to the INI
+file (in the *pyramid.includes* section) before running the *buildout* another time; don't forget to add the
+requested version at the end of *buildout.cfg* file, as Buildout is not configured by default to automatically
+download the last release of a given unknown package.
+
+*development.ini* and *production.ini* files contain many commented directives related to PyAMS components. Read and
+update them carefully before initializing your application database!
+
+
+Initializing the database
+-------------------------
+
+When you have downloaded and installed all required packages, you have to initialize the database so that all
+required components are available.
+
+From a shell, just type:
+
+.. code-block:: bash
+
+ (env) # ./bin/pyams_upgrade etc/development.ini
+
+This process requires that every package is correctly included into *pyramid.includes* directive from selected
+configuration file.
+
+
+Initializing Elasticsearch index
+--------------------------------
+
+If you want to use an Elasticsearch index, you have to initialize index settings and mappings; the Ingest attachment
+plug-in is also required to handle attachments correctly.
+
+Elasticsearch integration is defined through the *PyAMS_content_es* package. Configuration files are available in this
+package, for attachment pipeline, index settings and mappings:
+
+.. code-block:: bash
+
+ (env) # cd /var/local/src/pyams/pyams_content_es
+ (env) # curl --noproxy localhost -XDELETE http://localhost:9200/pyams (1)
+ (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams -d @index-settings.json
+
+ (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfNewsEvent/_mapping -d @mappings/WfNewsEvent.json
+ (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfTopic/_mapping -d @mappings/WfTopic.json
+ (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfBlogPost/_mapping -d @mappings/WfBlogPost.json
+
+(1) If 'pyams' is defined as Elasticsearch index name.
+
+
+NLTK initialization
+-------------------
+
+Some NLTK (Natural Language Toolkit) tokenizers and stopwords utilities are used to index fulltext contents elements.
+This package requires downloading and configuration of several elements which are done as follow:
+
+.. code-block:: bash
+
+ (end) # ./bin/py
+ >>> import nltk
+ >>> nltk.download()
+ NLTK Downloader
+ ---------------------------------------------------------------------------
+ d) Download l) List u) Update c) Config h) Help q) Quit
+ ---------------------------------------------------------------------------
+ Downloader> c
+
+ Data Server:
+ - URL: <https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml>
+ - 6 Package Collections Available
+ - 107 Individual Packages Available
+
+ Local Machine:
+ - Data directory: /home/tflorac/nltk_data
+
+ ---------------------------------------------------------------------------
+ s) Show Config u) Set Server URL d) Set Data Dir m) Main Menu
+ ---------------------------------------------------------------------------
+ Config> d
+ New directory> /usr/local/lib/nltk_data (1)
+ Config> m
+
+ ---------------------------------------------------------------------------
+ d) Download l) List u) Update c) Config h) Help q) Quit
+ ---------------------------------------------------------------------------
+ Downloader> d
+
+ Download which package (l=list; x=cancel)?
+ Identifier> punkt
+ Downloading package punkt to /usr/local/lib/nltk_data...
+
+ Downloader> d
+
+ Download which package (l=list; x=cancel)?
+ Identifier> stopwords
+ Downloading package stopwords to /usr/local/lib/nltk_data...
+
+
+(1) On Debian GNU/Linux, you can choose any directory between '*~/nltk_data*' (where '~' is the homedir of user running
+Pyramid application), '*/usr/share/nltk_data*', '*/usr/local/share/nltk_data*', '*/usr/lib/nltk_data*' and
+'*/usr/local/lib/nltk_data*'.
+
+
+Starting the application
+------------------------
+
+When database upgrade process has ended, you can start the web application process with the standard Pyramid's
+*pserve* command line tool:
+
+.. code-block:: bash
+
+ (env) # ./bin/pserve etc/development.ini
+
+In standard debug mode, all registered components are displayed in the console, until the final line (here using ZEO):
+
+.. code-block:: bash
+
+ 2018-01-14 11:37:54,339 INFO [ZEO.ClientStorage][MainThread] [('127.0.0.1', 8100)] ClientStorage (pid=28695) created RW/normal for storage: 'pyams'
+ 2018-01-14 11:37:54,340 INFO [ZEO.cache][MainThread] created temporary cache file 3
+ 2018-01-14 11:37:54,345 INFO [ZODB.blob][MainThread] (28695) Blob directory `/var/local/env/pyams/var/db/blobs` is used but has no layout marker set. Selected `lawn` layout.
+ 2018-01-14 11:37:54,345 WARNI [ZODB.blob][MainThread] (28695) The `lawn` blob directory layout is deprecated due to scalability issues on some file systems, please consider migrating to the `bushy` layout.
+ 2018-01-14 11:37:54,346 DEBUG [asyncio][[('127.0.0.1', 8100)] zeo client networking thread] Using selector: EpollSelector
+ 2018-01-14 11:37:54,347 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] disconnected <ZEO.asyncio.client.Client object at 0x7feeb1de7390> None
+ 2018-01-14 11:37:54,348 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] try_connecting
+ 2018-01-14 11:37:54,349 INFO [ZEO.asyncio.base][[('127.0.0.1', 8100)] zeo client networking thread] Connected Protocol(('127.0.0.1', 8100), 'pyams', False)
+ 2018-01-14 11:37:54,355 INFO [ZEO.ClientStorage][[('127.0.0.1', 8100)] zeo client networking thread] [('127.0.0.1', 8100)] Connected to storage: ('localhost', 8100)
+ 2018-01-14 11:37:54,358 DEBUG [txn.140663320073984][MainThread] new transaction
+ 2018-01-14 11:37:54,360 DEBUG [txn.140663320073984][MainThread] commit
+ 2018-01-14 11:37:54,484 DEBUG [config][MainThread] include /home/tflorac/Dropbox/src/PyAMS/pyams_template/src/pyams_template/configure.zcml
+ 2018-01-14 11:37:54,485 DEBUG [config][MainThread] include /var/local/env/pycharm/lib/python3.5/site-packages/pyramid_zcml/configure.zcml
+ ...
+ 2018-01-14 11:37:54,833 DEBUG [PyAMS (utils)][MainThread] Registering utility <class 'pyams_utils.timezone.utility.TimezoneGenerationsChecker'> named 'PyAMS timezone' providing <InterfaceClass pyams_utils.interfaces.site.ISiteGenerations>
+ 2018-01-14 11:37:54,834 DEBUG [PyAMS (utils)][MainThread] Registering class <class 'pyams_utils.timezone.vocabulary.TimezonesVocabulary'> as vocabulary with name "PyAMS timezones"
+ 2018-01-14 11:37:54,835 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.traversing.PathElementsAdapter'> for (<InterfaceClass zope.location.interfaces.IContained>,) providing <InterfaceClass pyams_utils.interfaces.traversing.IPathElements>
+ 2018-01-14 11:37:54,839 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.url.AbsoluteUrlTalesExtension'> for (<InterfaceClass zope.interface.Interface>, <InterfaceClass zope.interface.Interface>, <InterfaceClass zope.interface.Interface>) providing <InterfaceClass pyams_utils.interfaces.tales.ITALESExtension>
+ 2018-01-14 11:37:54,847 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.widget.decimal.DottedDecimalDataConverter'> for (<InterfaceClass pyams_utils.schema.IDottedDecimalField>, <InterfaceClass z3c.form.interfaces.IWidget>) providing <InterfaceClass z3c.form.interfaces.IDataConverter>
+ 2018-01-14 11:37:54,942 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.zmi.intids.IntIdsLengthAdapter'> for (<InterfaceClass zope.intid.interfaces.IIntIds>,) providing <InterfaceClass pyams_utils.interfaces.intids.IIndexLength>
+ 2018-01-14 11:37:54,943 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "properties.html" for <InterfaceClass zope.intid.interfaces.IIntIds> (<class 'pyams_utils.zmi.intids.IntIdsPropertiesDisplayForm'>)
+ 2018-01-14 11:37:54,949 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "properties.html" for <InterfaceClass pyams_utils.interfaces.timezone.IServerTimezone> (<class 'pyams_utils.zmi.timezone.ServerTimezonePropertiesEditForm'>)
+ 2018-01-14 11:37:54,980 DEBUG [PyAMS (utils)][MainThread] Registering class <class 'pyams_utils.zodb.ZEOConnectionVocabulary'> as vocabulary with name "PyAMS ZEO connections"
+ 2018-01-14 11:37:54,981 DEBUG [PyAMS (utils)][MainThread] Registering class <class 'pyams_utils.zodb.ZODBConnectionVocabulary'> as vocabulary with name "PyAMS ZODB connections"
+ 2018-01-14 11:37:55,015 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "add-zeo-connection.html" for <InterfaceClass zope.component.interfaces.ISite> (<class 'pyams_utils.zmi.zeo.ZEOConnectionAddForm'>)
+ 2018-01-14 11:37:55,016 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.zmi.zeo.ZEOConnectionNameAdapter'> for (<InterfaceClass pyams_utils.interfaces.zeo.IZEOConnection>, <InterfaceClass pyams_zmi.layer.IAdminLayer>) providing <InterfaceClass pyams_skin.interfaces.container.ITableElementName>
+ 2018-01-14 11:37:55,017 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "properties.html" for <InterfaceClass pyams_utils.interfaces.zeo.IZEOConnection> (<class 'pyams_utils.zmi.zeo.ZEOConnectionPropertiesEditForm'>)
+ ...
+ 2018-01-14 11:41:13,214 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_default_theme.skin.ResourcesAdapter'> for (<InterfaceClass zope.interface.Interface>, <InterfaceClass pyams_default_theme.layer.IPyAMSDefaultLayer>, <InterfaceClass zope.interface.Interface>) providing <InterfaceClass pyams_skin.interfaces.resources.IResources>
+ 2018-01-14 11:43:36,665 INFO [ZEO.ClientStorage][MainThread] [('127.0.0.1', 8100)] ClientStorage (pid=29335) created RW/normal for storage: 'pyams'
+ 2018-01-14 11:43:36,665 INFO [ZEO.cache][MainThread] created temporary cache file 9
+ 2018-01-14 11:43:36,673 DEBUG [asyncio][[('127.0.0.1', 8100)] zeo client networking thread] Using selector: EpollSelector
+ 2018-01-14 11:43:36,674 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 Checking blob cache size. (target: 966367642)
+ 2018-01-14 11:43:36,674 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] disconnected <ZEO.asyncio.client.Client object at 0x7ffa54058860> None
+ 2018-01-14 11:43:36,675 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 blob cache size: 0
+ 2018-01-14 11:43:36,675 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] try_connecting
+ 2018-01-14 11:43:36,675 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 -->
+ 2018-01-14 11:43:36,677 INFO [ZEO.asyncio.base][[('127.0.0.1', 8100)] zeo client networking thread] Connected Protocol(('127.0.0.1', 8100), 'pyams', False)
+ 2018-01-14 11:43:36,679 INFO [ZEO.ClientStorage][[('127.0.0.1', 8100)] zeo client networking thread] [('127.0.0.1', 8100)] Connected to storage: ('localhost', 8100)
+ 2018-01-14 11:43:36,682 DEBUG [txn.140713340237568][MainThread] new transaction
+ 2018-01-14 11:43:36,683 DEBUG [txn.140713340237568][MainThread] commit
+ 2018-01-14 11:43:36,690 INFO [PyAMS (scheduler][MainThread] Starting tasks scheduler <SchedulerProcess(SchedulerProcess-1, initial)>...
+ 2018-01-14 11:43:36,698 INFO [PyAMS (scheduler][MainThread] Started tasks scheduler with PID 29361.
+ 2018-01-14 11:43:36,701 INFO [apscheduler.scheduler][MainThread] Scheduler started
+ 2018-01-14 11:43:36,702 DEBUG [apscheduler.scheduler][APScheduler] Looking for jobs to run
+ 2018-01-14 11:43:36,704 DEBUG [apscheduler.scheduler][APScheduler] No jobs; waiting until a job is added
+ 2018-01-14 11:43:36,719 INFO [ZEO.ClientStorage][MainThread] [('127.0.0.1', 8100)] ClientStorage (pid=29335) created RW/normal for storage: 'pyams'
+ 2018-01-14 11:43:36,720 INFO [ZEO.cache][MainThread] created temporary cache file 15
+ 2018-01-14 11:43:36,724 DEBUG [asyncio][[('127.0.0.1', 8100)] zeo client networking thread] Using selector: EpollSelector
+ 2018-01-14 11:43:36,725 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] disconnected <ZEO.asyncio.client.Client object at 0x7ffa557e8b00> None
+ 2018-01-14 11:43:36,726 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] try_connecting
+ 2018-01-14 11:43:36,727 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 Checking blob cache size. (target: 966367642)
+ 2018-01-14 11:43:36,728 INFO [ZEO.asyncio.base][[('127.0.0.1', 8100)] zeo client networking thread] Connected Protocol(('127.0.0.1', 8100), 'pyams', False)
+ 2018-01-14 11:43:36,729 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 blob cache size: 0
+ 2018-01-14 11:43:36,729 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 -->
+ 2018-01-14 11:43:36,732 INFO [ZEO.ClientStorage][[('127.0.0.1', 8100)] zeo client networking thread] [('127.0.0.1', 8100)] Connected to storage: ('localhost', 8100)
+ 2018-01-14 11:43:36,735 DEBUG [txn.140713340237568][MainThread] new transaction
+ 2018-01-14 11:43:36,736 DEBUG [txn.140713340237568][MainThread] commit
+ 2018-01-14 11:43:36,743 INFO [PyAMS (media)][MainThread] Starting medias converter <MediaConversionProcess(MediaConversionProcess-2, initial)>...
+ 2018-01-14 11:43:36,751 INFO [PyAMS (media)][MainThread] Started medias converter with PID 29367.
+ Starting server in PID 29335.
+ Serving on http://0.0.0.0:6543
+
+
+From this point, you can launch a browser and open URL *http://127.0.0.1:6543/admin* to get access to PyAMS
+management interface; default login is "admin/admin", that you may change as soon as possible (see
+:ref:`pyams_security`)!!.
--- a/src/source/architecture.rst Mon Mar 05 10:40:40 2018 +0100
+++ b/src/source/architecture.rst Tue Mar 06 11:06:45 2018 +0100
@@ -2,3 +2,8 @@
PyAMS application architecture
==============================
+
+.. toctree::
+ :maxdepth: 2
+
+ site
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/src/source/components.rst Tue Mar 06 11:06:45 2018 +0100
@@ -0,0 +1,10 @@
+.. _components:
+
+Custom PyAMS components
+=======================
+
+.. toctree::
+ :maxdepth: 2
+
+ traverser
+ tales
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/src/source/extend.rst Tue Mar 06 11:06:45 2018 +0100
@@ -0,0 +1,10 @@
+.. _extending:
+
+Extending PyAMS
+===============
+
+.. toctree::
+ :maxdepth: 2
+
+ zca
+ components
--- a/src/source/index.rst Mon Mar 05 10:40:40 2018 +0100
+++ b/src/source/index.rst Tue Mar 06 11:06:45 2018 +0100
@@ -17,22 +17,22 @@
PyAMS is built on top of MyAMS (My Application Management Skin), a small web client framework built on top of JQuery
and Bootstrap, which was developed by the french national forestry office (ONF -- Office national des forêts --
-http://www.onf.fr) to build Java and Python web applications. The new ONF website is now completely handled with
-PyAMS framework.
+http://www.onf.fr) to build web applications in several languages (actually java, Python and PHP). The new ONF website
+is now completely handled with PyAMS framework.
PyAMS is a multipurpose set of packages, providing tools including:
* custom interfaces
-* custom ZCA registry annotations
+* custom registry annotations
+* custom security policy
* local registry support
* network protocols utilities (for HTTP and XML-RPC)
* command line scripts
-* a custom security policy
-* custom utilities...
+* custom utilities.
If you just want to know how easy it is to create a PyAMS application instance, just read the :ref:`quickstart`!
-To know how PyAMS management interface looks like, just see :ref:`screenshots`!
+To know how PyAMS management interface looks like, look at :ref:`screenshots`!
.. toctree::
@@ -40,13 +40,10 @@
quickstart
architecture
- zodb
install
- zca
- site
- traverser
- tales
- utilities
+ config
+ manage
+ extend
PyAMS applications architecture
--- a/src/source/install.rst Mon Mar 05 10:40:40 2018 +0100
+++ b/src/source/install.rst Tue Mar 06 11:06:45 2018 +0100
@@ -3,379 +3,8 @@
Installing PyAMS
================
-PyAMS default installation is based on `Buildout <http://www.buildout.org>`_ utility. It's not mandatory to use a
-virtual environment, but it allows you to have a better control over your Python resources.
-
-Current PyAMS version is based and validated for Python 3.5; your Python environment must also include a C
-compiler as well as development headers for Python, *libjpeg*, *libpng*, *libfreetype*, *libxml2*, *libxslt* and
-eventually *libldap*, *libpq*, *libffi*, *libgdal* or *libzmq*. *Cython* compiler can also be useful to optimize
-several packages.
-
-PyAMS default components configuration also pre-suppose that the following external tools are available:
-
-- a *Memcached* or *Redis* server, to store sessions and cache (can be changed through Beaker configuration)
-
-Optional tools also include:
-
-- an *LDAP* server for authentication
-
-- an *ElasticSearch* server for full text indexing (see *PyAMS_content_es* package)
-
-- a *WebSockets* server using AsyncIO. This is used to manage notifications (see *PyAMS_notify* and *PyAMS_notify_ws*
- packages). An *out of the box* environment can be built using *pyams_notify* scaffold.
-
-
-PyAMS also needs that you use a ZODB shared server, as several background processes needing a concurrent access to ZODB
-are started by PyAMS main process. Three ZODB storages are already provided through PyAMS: ZEO, RelStorage or Newt.db.
-See :ref:`zodb` to know how to initialize database with the help of PyAMS tools.
-
-
-Creating initial buildout
--------------------------
-
-PyAMS provides a new Pyramid scaffold, called *pyams*, generated via a *cookiecutter* template.
-
-A simple option to install PyAMS is to create a buildout environment including *Pyramid* and all *PyAMS* packages:
-
-.. code-block:: bash
-
- # mkdir /var/local/
- # pip3 install virtualenv
- # virtualenv --python=python3.5 env
- # cd env
- # . bin/activate
- (env) # pip3.5 install cookiecutter
- (env) # cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/pyams
-
-*CookieCutter* will ask you for a small set of input variables that you can change or not:
-
-- **pyams_release**: version of PyAMS configuration file to use. "latest" (default value) will point to last release;
- you can also choose to point to a given release ("0.1.4" for example)
-
-- **project_name**: current environment name in "human form"
-
-- **project_slug**: "technical" package name, based on project name
-
-- **virtual_hostname**: Apache virtual-host name
-
-- **webapp_name**: web application package name ("webapp" as default)
-
-- **webapp_port**: TCP/IP port to use when running application outside Apache ("6543" as default)
-
-- **eggs_directory**: relative or absolute path to directory containing downloaded eggs; this directory can be
- shared with other projects ("eggs" as default)
-
-- **logs_directory**: absolute path to directory containing Apache's log files
-
-- **run_user**: user name under which Apache process will run ("www-data" as default)
-
-- **run_group**: group name under which Apache process will run ("www-data" as default)
-
-- **beaker_backend**: name of Beaker backend to use to store sessions and cache data ("redis" as default)
-
-- **beaker_server**: IP address and port of Beaker backend server ("127.0.0.1:6379" as default)
-
-- **db_type**: ZODB database storage; available options include ZEO, RelStorage and NewtDB
-
-- **db_host**: IP address of database server ("127.0.0.1" as default); WARNING: database server installation
- is not part of application installation; another "zeo_server" cookiecutter recipe is available for ZEO
-
-- **db_port**: listening port of database server ("8100" is given as default for ZEO)
-
-- **db_name**: database or ZEO storage name to use
-
-- **db_username**: database user name
-
-- **db_password**: database password
-
-- **zeo_realm**: ZEO authentication realm
-
-- **blobs_dir**: local directory to use to store cache of ZODB blobs; cache size is limited to 10GB as default
-
-- **use_postgresql**: specify if PostgreSQL access is required; if so, please check that PostgreSQL development files
- are available to compile PsycoPG2 extension
-
-- **use_oracle**: specify if Oracle access is required; if so, please check that Oracle development files are
- available to compile cx_Oracle extension, and that ORACLE_HOME environment variable is correctly defined (see below)
-
-- **use_ldap**: specify if LDAP access will be required for authentication
-
-- **use_elasticsearch**: specify if an ElasticSearch server will be used for indexation
-
-- **elasticsearch_server**: URL used to access Elasticsearch server ("http://127.0.0.1:9200" as default); this URL can
- include login and password ("http://login:password@127.0.0.1:9200"), if required...
-
-- **elasticsearch_index**: name of Elasticsearch index to use ("pyams" as default)
-
-- **create_elasticsearch_index**: specify if Elasticsearch index should be created after installation is complete
-
-- **define_elasticsearch_mappings** : specify if Elasticsearch mappings should be defined after installation is complete
-
-- **smtp_server**: DNS name of SMTP server ("localhost" as default)
-
-- **smtp_server_name**: "human" name given to SMTP server ("pyams" as default)
-
-- **pyams_scheduler**: TCP/IP address and port to use to access PyAMS tasks scheduler process ("127.0.0.1:5555" as
- default); see :ref:`pyams_scheduler`
-
-- **start_scheduler**: boolean value to indicate if scheduler process is started by this application instance
-
-- **pyams_medias_converter**: TCP/IP address and port to use to access PyAMS medias converter process ("127.0.0.1:5556"
- as default); see :ref:`pyams_medias`
-
-- **start_medias_converter**: boolean value to indicate if medias converter process is started by this application
- instance
-
-- **pyams_es_indexer**: TCP/IP address and port to use to access PyAMS Elasticsearch indexer process ("127.0.0.1:5557"
- as default); see :ref:`pyams_content_es`
-
-- **start_es_indexer** boolean value to indicate if Elasticsearch indexer process is started by this application
- instance
-
-- **use_notifications**: specify if PyAMS notifications services are to be used (see :ref:`pyams_notify`)
-
-- **pyams_ws_notify**: TCP/IP address and port of PyAMS websockets server managing notifications service
- ("127.0.0.1:8081" as default)
-
-- **lexicon_languages**: NLTK lexicon languages to use ("en:english fr:french" as default)
-
-- **extension_package**: name of a PyAMS extension package to include in environment configuration
-
-- **need_pyams_gis**: specify if PyAMS GIS features are to be used by given extension package; if so, please check
- that *libgdal* development files are available; on Debian (and maybe others), you have to specify environment
- variables (see below).
-
-
-Boolean values accept "true", "yes", "on" or "1" (in any case) as *true* values, and anything else as *false*; setting
-any option to empty string will keep the default proposed value.
-
-You can then check, and eventually update, the proposed Buildout configuration file *buildout.cfg*, to add or remove
-packages or update settings to your needs. Then finalize Bootstrap initialization:
-
-.. code-block:: bash
-
- (env) # python3.5 bootstrap.py
- (env) # ./bin/buildout
-
-This last operation can be quite long, as many packages have to downloaded, compiled and installed in the virtual
-environment. If you encounter any compile error, just install the required dependencies and restart the buildout.
-
-Some dependencies can require the definition of custom environment variables before running *buildout*, like:
-
-- for *libgdal*, which is required by **PyAMS_gis** package, use:
-
-.. code-block:: bash
-
- (env) # export C_INCLUDE_PATH=/usr/include/gdal
- (env) # export CPLUS_INCLUDE_PATH=/usr/include/gdal
-
-**WARNING**: you have to check also that your *libgdal* release is matching "GDAL" release given in PyAMS
-configuration file (actually 2.1.0).
-
-- for *cx_Oracle*, which is required if you use Oracle database connections, use:
-
-.. code-block:: bash
-
- (env) # export ORACLE_HOME=/usr/lib/oracle/12.1/client64
-
-These examples are given for Debian GNU/Linux. You may have to adapt configuration based on your own Linux
-distribution and packages versions.
-
-
-Environment settings
---------------------
-
-The project generated from *pyams* scaffold is based on default Pyramid's *zodb* scaffold, but it adds:
-
-- a custom application factory, in the *webapp* directory (see :ref:`site`)
+.. toctree::
+ :maxdepth: 2
-- a set of directories to store runtime data, in the *var* directory; each directory contains a *README.txt* file
- which should be self-explanatory to indicate what this directory should contain, including a ZEO cache
-
-- a set of configuration files, in the *etc* directory; here are standard *development.ini* and *production.ini*
- configuration files, a ZODB configuration files (*zodb-zeo.conf*) for a ZEO client storage and two Apache
- configurations (for Apache 2.2 and 2.4) using *mod_wsgi*.
-
-Once the project have been created from the scaffold, you are free to update all the configuration files.
-
-If you need to add packages to the environment, you have to add them to the *buildout.cfg* file **AND** to the INI
-file (in the *pyramid.includes* section) before running the *buildout* another time; don't forget to add the
-requested version at the end of *buildout.cfg* file, as Buildout is not configured by default to automatically
-download the last release of a given unknown package.
-
-*development.ini* and *production.ini* files contain many commented directives related to PyAMS components. Read and
-update them carefully before initializing your application database!
-
-
-Initializing the database
--------------------------
-
-When you have downloaded and installed all required packages, you have to initialize the database so that all
-required components are available.
-
-From a shell, just type:
-
-.. code-block:: bash
-
- (env) # ./bin/pyams_upgrade etc/development.ini
-
-This process requires that every package is correctly included into *pyramid.includes* directive from selected
-configuration file.
-
-
-Initializing Elasticsearch index
---------------------------------
-
-If you want to use an Elasticsearch index, you have to initialize index settings and mappings; the Ingest attachment
-plug-in is also required to handle attachments correctly.
-
-Elasticsearch integration is defined through the *PyAMS_content_es* package. Configuration files are available in this
-package, for attachment pipeline, index settings and mappings:
-
-.. code-block:: bash
-
- (env) # cd /var/local/src/pyams/pyams_content_es
- (env) # curl --noproxy localhost -XDELETE http://localhost:9200/pyams (1)
- (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams -d @index-settings.json
-
- (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfNewsEvent/_mapping -d @mappings/WfNewsEvent.json
- (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfTopic/_mapping -d @mappings/WfTopic.json
- (env) # curl --noproxy localhost -XPUT http://localhost:9200/pyams/WfBlogPost/_mapping -d @mappings/WfBlogPost.json
-
-(1) If 'pyams' is defined as Elasticsearch index name.
-
-
-NLTK initialization
--------------------
-
-Some NLTK (Natural Language Toolkit) tokenizers and stopwords utilities are used to index fulltext contents elements.
-This package requires downloading and configuration of several elements which are done as follow:
-
-.. code-block:: bash
-
- (end) # ./bin/py
- >>> import nltk
- >>> nltk.download()
- NLTK Downloader
- ---------------------------------------------------------------------------
- d) Download l) List u) Update c) Config h) Help q) Quit
- ---------------------------------------------------------------------------
- Downloader> c
-
- Data Server:
- - URL: <https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml>
- - 6 Package Collections Available
- - 107 Individual Packages Available
-
- Local Machine:
- - Data directory: /home/tflorac/nltk_data
-
- ---------------------------------------------------------------------------
- s) Show Config u) Set Server URL d) Set Data Dir m) Main Menu
- ---------------------------------------------------------------------------
- Config> d
- New directory> /usr/local/lib/nltk_data (1)
- Config> m
-
- ---------------------------------------------------------------------------
- d) Download l) List u) Update c) Config h) Help q) Quit
- ---------------------------------------------------------------------------
- Downloader> d
-
- Download which package (l=list; x=cancel)?
- Identifier> punkt
- Downloading package punkt to /usr/local/lib/nltk_data...
-
- Downloader> d
-
- Download which package (l=list; x=cancel)?
- Identifier> stopwords
- Downloading package stopwords to /usr/local/lib/nltk_data...
-
-
-(1) On Debian GNU/Linux, you can choose any directory between '*~/nltk_data*' (where '~' is the homedir of user running
-Pyramid application), '*/usr/share/nltk_data*', '*/usr/local/share/nltk_data*', '*/usr/lib/nltk_data*' and
-'*/usr/local/lib/nltk_data*'.
-
-
-Starting the application
-------------------------
-
-When database upgrade process has ended, you can start the web application process with the standard Pyramid's
-*pserve* command line tool:
-
-.. code-block:: bash
-
- (env) # ./bin/pserve etc/development.ini
-
-In standard debug mode, all registered components are displayed in the console, until the final line (here using ZEO):
-
-.. code-block:: bash
-
- 2018-01-14 11:37:54,339 INFO [ZEO.ClientStorage][MainThread] [('127.0.0.1', 8100)] ClientStorage (pid=28695) created RW/normal for storage: 'pyams'
- 2018-01-14 11:37:54,340 INFO [ZEO.cache][MainThread] created temporary cache file 3
- 2018-01-14 11:37:54,345 INFO [ZODB.blob][MainThread] (28695) Blob directory `/var/local/env/pyams/var/db/blobs` is used but has no layout marker set. Selected `lawn` layout.
- 2018-01-14 11:37:54,345 WARNI [ZODB.blob][MainThread] (28695) The `lawn` blob directory layout is deprecated due to scalability issues on some file systems, please consider migrating to the `bushy` layout.
- 2018-01-14 11:37:54,346 DEBUG [asyncio][[('127.0.0.1', 8100)] zeo client networking thread] Using selector: EpollSelector
- 2018-01-14 11:37:54,347 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] disconnected <ZEO.asyncio.client.Client object at 0x7feeb1de7390> None
- 2018-01-14 11:37:54,348 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] try_connecting
- 2018-01-14 11:37:54,349 INFO [ZEO.asyncio.base][[('127.0.0.1', 8100)] zeo client networking thread] Connected Protocol(('127.0.0.1', 8100), 'pyams', False)
- 2018-01-14 11:37:54,355 INFO [ZEO.ClientStorage][[('127.0.0.1', 8100)] zeo client networking thread] [('127.0.0.1', 8100)] Connected to storage: ('localhost', 8100)
- 2018-01-14 11:37:54,358 DEBUG [txn.140663320073984][MainThread] new transaction
- 2018-01-14 11:37:54,360 DEBUG [txn.140663320073984][MainThread] commit
- 2018-01-14 11:37:54,484 DEBUG [config][MainThread] include /home/tflorac/Dropbox/src/PyAMS/pyams_template/src/pyams_template/configure.zcml
- 2018-01-14 11:37:54,485 DEBUG [config][MainThread] include /var/local/env/pycharm/lib/python3.5/site-packages/pyramid_zcml/configure.zcml
- ...
- 2018-01-14 11:37:54,833 DEBUG [PyAMS (utils)][MainThread] Registering utility <class 'pyams_utils.timezone.utility.TimezoneGenerationsChecker'> named 'PyAMS timezone' providing <InterfaceClass pyams_utils.interfaces.site.ISiteGenerations>
- 2018-01-14 11:37:54,834 DEBUG [PyAMS (utils)][MainThread] Registering class <class 'pyams_utils.timezone.vocabulary.TimezonesVocabulary'> as vocabulary with name "PyAMS timezones"
- 2018-01-14 11:37:54,835 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.traversing.PathElementsAdapter'> for (<InterfaceClass zope.location.interfaces.IContained>,) providing <InterfaceClass pyams_utils.interfaces.traversing.IPathElements>
- 2018-01-14 11:37:54,839 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.url.AbsoluteUrlTalesExtension'> for (<InterfaceClass zope.interface.Interface>, <InterfaceClass zope.interface.Interface>, <InterfaceClass zope.interface.Interface>) providing <InterfaceClass pyams_utils.interfaces.tales.ITALESExtension>
- 2018-01-14 11:37:54,847 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.widget.decimal.DottedDecimalDataConverter'> for (<InterfaceClass pyams_utils.schema.IDottedDecimalField>, <InterfaceClass z3c.form.interfaces.IWidget>) providing <InterfaceClass z3c.form.interfaces.IDataConverter>
- 2018-01-14 11:37:54,942 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.zmi.intids.IntIdsLengthAdapter'> for (<InterfaceClass zope.intid.interfaces.IIntIds>,) providing <InterfaceClass pyams_utils.interfaces.intids.IIndexLength>
- 2018-01-14 11:37:54,943 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "properties.html" for <InterfaceClass zope.intid.interfaces.IIntIds> (<class 'pyams_utils.zmi.intids.IntIdsPropertiesDisplayForm'>)
- 2018-01-14 11:37:54,949 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "properties.html" for <InterfaceClass pyams_utils.interfaces.timezone.IServerTimezone> (<class 'pyams_utils.zmi.timezone.ServerTimezonePropertiesEditForm'>)
- 2018-01-14 11:37:54,980 DEBUG [PyAMS (utils)][MainThread] Registering class <class 'pyams_utils.zodb.ZEOConnectionVocabulary'> as vocabulary with name "PyAMS ZEO connections"
- 2018-01-14 11:37:54,981 DEBUG [PyAMS (utils)][MainThread] Registering class <class 'pyams_utils.zodb.ZODBConnectionVocabulary'> as vocabulary with name "PyAMS ZODB connections"
- 2018-01-14 11:37:55,015 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "add-zeo-connection.html" for <InterfaceClass zope.component.interfaces.ISite> (<class 'pyams_utils.zmi.zeo.ZEOConnectionAddForm'>)
- 2018-01-14 11:37:55,016 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_utils.zmi.zeo.ZEOConnectionNameAdapter'> for (<InterfaceClass pyams_utils.interfaces.zeo.IZEOConnection>, <InterfaceClass pyams_zmi.layer.IAdminLayer>) providing <InterfaceClass pyams_skin.interfaces.container.ITableElementName>
- 2018-01-14 11:37:55,017 DEBUG [PyAMS (pagelet)][MainThread] Registering pagelet view "properties.html" for <InterfaceClass pyams_utils.interfaces.zeo.IZEOConnection> (<class 'pyams_utils.zmi.zeo.ZEOConnectionPropertiesEditForm'>)
- ...
- 2018-01-14 11:41:13,214 DEBUG [PyAMS (utils)][MainThread] Registering adapter <class 'pyams_default_theme.skin.ResourcesAdapter'> for (<InterfaceClass zope.interface.Interface>, <InterfaceClass pyams_default_theme.layer.IPyAMSDefaultLayer>, <InterfaceClass zope.interface.Interface>) providing <InterfaceClass pyams_skin.interfaces.resources.IResources>
- 2018-01-14 11:43:36,665 INFO [ZEO.ClientStorage][MainThread] [('127.0.0.1', 8100)] ClientStorage (pid=29335) created RW/normal for storage: 'pyams'
- 2018-01-14 11:43:36,665 INFO [ZEO.cache][MainThread] created temporary cache file 9
- 2018-01-14 11:43:36,673 DEBUG [asyncio][[('127.0.0.1', 8100)] zeo client networking thread] Using selector: EpollSelector
- 2018-01-14 11:43:36,674 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 Checking blob cache size. (target: 966367642)
- 2018-01-14 11:43:36,674 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] disconnected <ZEO.asyncio.client.Client object at 0x7ffa54058860> None
- 2018-01-14 11:43:36,675 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 blob cache size: 0
- 2018-01-14 11:43:36,675 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] try_connecting
- 2018-01-14 11:43:36,675 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 -->
- 2018-01-14 11:43:36,677 INFO [ZEO.asyncio.base][[('127.0.0.1', 8100)] zeo client networking thread] Connected Protocol(('127.0.0.1', 8100), 'pyams', False)
- 2018-01-14 11:43:36,679 INFO [ZEO.ClientStorage][[('127.0.0.1', 8100)] zeo client networking thread] [('127.0.0.1', 8100)] Connected to storage: ('localhost', 8100)
- 2018-01-14 11:43:36,682 DEBUG [txn.140713340237568][MainThread] new transaction
- 2018-01-14 11:43:36,683 DEBUG [txn.140713340237568][MainThread] commit
- 2018-01-14 11:43:36,690 INFO [PyAMS (scheduler][MainThread] Starting tasks scheduler <SchedulerProcess(SchedulerProcess-1, initial)>...
- 2018-01-14 11:43:36,698 INFO [PyAMS (scheduler][MainThread] Started tasks scheduler with PID 29361.
- 2018-01-14 11:43:36,701 INFO [apscheduler.scheduler][MainThread] Scheduler started
- 2018-01-14 11:43:36,702 DEBUG [apscheduler.scheduler][APScheduler] Looking for jobs to run
- 2018-01-14 11:43:36,704 DEBUG [apscheduler.scheduler][APScheduler] No jobs; waiting until a job is added
- 2018-01-14 11:43:36,719 INFO [ZEO.ClientStorage][MainThread] [('127.0.0.1', 8100)] ClientStorage (pid=29335) created RW/normal for storage: 'pyams'
- 2018-01-14 11:43:36,720 INFO [ZEO.cache][MainThread] created temporary cache file 15
- 2018-01-14 11:43:36,724 DEBUG [asyncio][[('127.0.0.1', 8100)] zeo client networking thread] Using selector: EpollSelector
- 2018-01-14 11:43:36,725 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] disconnected <ZEO.asyncio.client.Client object at 0x7ffa557e8b00> None
- 2018-01-14 11:43:36,726 DEBUG [ZEO.asyncio.client][[('127.0.0.1', 8100)] zeo client networking thread] try_connecting
- 2018-01-14 11:43:36,727 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 Checking blob cache size. (target: 966367642)
- 2018-01-14 11:43:36,728 INFO [ZEO.asyncio.base][[('127.0.0.1', 8100)] zeo client networking thread] Connected Protocol(('127.0.0.1', 8100), 'pyams', False)
- 2018-01-14 11:43:36,729 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 blob cache size: 0
- 2018-01-14 11:43:36,729 DEBUG [ZEO.ClientStorage.check_blob_cache][[('127.0.0.1', 8100)] zeo client check blob size thread] 140712483907328 -->
- 2018-01-14 11:43:36,732 INFO [ZEO.ClientStorage][[('127.0.0.1', 8100)] zeo client networking thread] [('127.0.0.1', 8100)] Connected to storage: ('localhost', 8100)
- 2018-01-14 11:43:36,735 DEBUG [txn.140713340237568][MainThread] new transaction
- 2018-01-14 11:43:36,736 DEBUG [txn.140713340237568][MainThread] commit
- 2018-01-14 11:43:36,743 INFO [PyAMS (media)][MainThread] Starting medias converter <MediaConversionProcess(MediaConversionProcess-2, initial)>...
- 2018-01-14 11:43:36,751 INFO [PyAMS (media)][MainThread] Started medias converter with PID 29367.
- Starting server in PID 29335.
- Serving on http://0.0.0.0:6543
-
-
-From this point, you can launch a browser and open URL *http://127.0.0.1:6543/admin* to get access to PyAMS
-management interface; default login is "admin/admin", that you may change as soon as possible (see
-:ref:`pyams_security`)!!.
+ zodb
+ appinstall
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/src/source/manage.rst Tue Mar 06 11:06:45 2018 +0100
@@ -0,0 +1,10 @@
+.. _manage:
+
+Managing PyAMS environment
+==========================
+
+.. toctree::
+ :maxdepth: 2
+
+ utilities
+
--- a/src/source/quickstart.rst Mon Mar 05 10:40:40 2018 +0100
+++ b/src/source/quickstart.rst Tue Mar 06 11:06:45 2018 +0100
@@ -10,6 +10,10 @@
See :ref:`install` to get a full description of installation process.
+.. contents::
+ :local:
+
+
To do first
+++++++++++
@@ -107,6 +111,9 @@
daemon process started, pid=26230
+See :ref:`zodb` to know more about ZODB storage alternatives.
+
+
Create application instance
+++++++++++++++++++++++++++
@@ -187,6 +194,8 @@
Packages download, compilation and installation can easily take several minutes. Install errors are generally due to
missing libraries development headers.
+See :ref:`appinstall` to get more detailed information about installation process.
+
Start application
+++++++++++++++++
--- a/src/source/scripts.rst Mon Mar 05 10:40:40 2018 +0100
+++ b/src/source/scripts.rst Tue Mar 06 11:06:45 2018 +0100
@@ -2,3 +2,4 @@
PyAMS maintenance scripts
=========================
+
--- a/src/source/zodb.rst Mon Mar 05 10:40:40 2018 +0100
+++ b/src/source/zodb.rst Tue Mar 06 11:06:45 2018 +0100
@@ -148,3 +148,48 @@
Database creation is done as with RelStorage, but NewtDB add several schema objects. Migration scripts are available
if you need to switch from a classic RelStorage database to a Newt database.
+
+
+.. _ZODB.init:
+
+Initializing ZODB
++++++++++++++++++
+
+When your ZODB is created, you have to initialize it!
+
+After installing PyAMS application (see :ref:`appinstall`), a database upgrade script is available which allows you
+to initialize the database. See :ref:`config` and :ref:`scripts` to know more.
+
+
+.. _ZODB.migration:
+
+ZODB migration
+++++++++++++++
+
+After installation, you can switch from a given storage to another one with the help of the *zodbconvert* command
+line script provided by RelStorage.
+
+This Python script is using a configuration file containing directives of both source and target databases, which can
+be any storage described in the previous sections (or can even use the same storage).
+
+Here is a sample configuration file to convert a ZODB from a ZEO to RelStorage:
+
+.. code-block::
+
+ %import relstorage
+
+ <zeoclient source>
+ server zeo-server.mydomain:8100
+ storage pyams
+ blob-dir /var/local/env/zeo/var/zeoclient/blobs
+ shared-blob-dir false
+ </zeoclient>
+
+ <relstorage destination>
+ keep-history false
+ blob-dir /var/local/env/pyams/var/relstorage/blobs
+ shared-blob-dir false
+ <postgresql>
+ dsn host='postgresql-server.mydomain' dbname='pyams_rs' user='pyams' password='xxxxxxxx'
+ </postgresql>
+ </relstorage>