Mercurial Mercurial > pyams > pyams_user_guide / comparison
summary | shortlog | changelog | graph | tags | bookmarks | branches | files | changeset | file | latest | revisions | annotate | diff | comparison | raw | help
Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
src/source/plugins.rst
branchdoc-dc
changeset 56 60a1fbdbbed3
parent 55 949d496c4e96
child 57 e7d62e94392f
equal deleted inserted replaced
55:949d496c4e96 56:60a1fbdbbed3 
    11 plug-in is also required to handle attachments correctly.
 
    11 plug-in is also required to handle attachments correctly.
 
    12 
    12 
    13 Visit  https://www.elastic.co/ to learn how to install Elasticsearch Server, and how install `ingest-attachment` plug-in
 
    13 Visit  https://www.elastic.co/ to learn how to install Elasticsearch Server, and how install `ingest-attachment` plug-in
 
    14 
    14 
    15 
    15 
    16 .. tips:: Documentation for installing ElasticSearch 5.4
 
    16 .. tip:: Documentation for installing ElasticSearch 5.4
 
    17 
    17 
    18     - https://www.elastic.co/guide/en/elasticsearch/reference/5.4/gs-installation.html
 
    18     - https://www.elastic.co/guide/en/elasticsearch/reference/5.4/gs-installation.html
 
    19     - https://www.elastic.co/guide/en/elasticsearch/plugins/5.4/ingest-attachment.html
 
    19     - https://www.elastic.co/guide/en/elasticsearch/plugins/5.4/ingest-attachment.html
 
    20 
    20 
    21 
    21 
    77 
    77 
    78 
    78 
    79 3. Create or update index:
 
    79 3. Create or update index:
 
    80 ''''''''''''''''''''''''''
 
    80 ''''''''''''''''''''''''''
 
    81 
    81 
    82 You have to index PyAMS objects into ES database. From a shell:
 
    82 You have to Update ElasticSearch indexes with all database contents with ``pymas_es_index``. From a shell:
 
    83 
    83 
    84 .. code-block:: bash
 
    84 .. code-block:: bash
 
    85 
    85 
    86     (env) $ ./bin/pyams_es_index ../etc/development.ini
 
    86     (env) $ ./bin/pyams_es_index ../etc/development.ini
 
    87 
    87 
    91 
    91 
    92 Natural Language Toolkit - NLTK
 
    92 Natural Language Toolkit - NLTK
 
    93 ===============================
 
    93 ===============================
 
    94 
    94 
    95 
    95 
    96 With the package *PyAMS_nltk* PyAMS can use the NLTK features
 
    96 PyAMS enjoy the NLTK features through the *PyAMS_calalog*
 
    97 
    97 
    98 .. seealso::
 
    98 .. seealso::
 
    99 
    99 
   100     Visit https://www.nltk.org/ to learn more about NLTK
 
   100     Visit https://www.nltk.org/ to learn more about NLTK
 
   101 
   101 
   102 
   102 
   103 
   103 
       
   104 Initializing NLTK (Natural Language Toolkit)
 
       
   105 --------------------------------------------
 
   104 
   106 
   105 Initializing NLTK
 
   107 Some NLTK Collections like **tokenizers** and **stopwords** utilities are used to index fulltext contents
 
   106 -----------------
 
   108 elements. You can enhanced NTKL indexation according to your own needs. This package requires downloading and
 
   107 
   109 configuration of several elements which are done as follow:
 
   108 Some NLTK (Natural Language Toolkit) tokenizers and stopwords utilities are used to index fulltext contents elements.
 
       
   109 This package requires downloading and configuration of several elements which are done as follow:
 
       
   110 
   110 
   111 
   111 
   112 *1. Run the Python shell with PyAMS environment:*
 
   112 *1. Run the Python shell with PyAMS environment:*
 
   113 
   113 
   114 .. code-block:: bash
 
   114 .. code-block:: bash
 
   116     (env) $ ./bin/py
 
   116     (env) $ ./bin/py
 
   117 
   117 
   118 
   118 
   119 *2. In the Python shell:*
 
   119 *2. In the Python shell:*
 
   120 
   120 
   121 .. code-block:: python
 
   121 .. code-block:: pycon
 
   122 
   122 
   123     >>> import nltk
 
   123     >>> import nltk
 
   124     >>> nltk.download()
 
   124     >>> nltk.download()
 
   125 
   125 
   126 .. code-block:: python
 
   126 
       
   127 *3. Configuration installation directory:*
 
       
   128 
       
   129 .. tip::
 
       
   130 
       
   131     On Debian GNU/Linux, you can choose any directory between '*~/nltk_data*' (where '~' is the homedir of user running
 
       
   132     Pyramid application), '*/usr/share/nltk_data*', '*/usr/local/share/nltk_data*', '*/usr/lib/nltk_data*' and
 
       
   133     '*/usr/local/lib/nltk_data*'
 
       
   134 
       
   135     Please check if you have permission to write to this directory
 
       
   136 
       
   137 
       
   138 .. code-block:: shell
 
   127 
   139 
   128     NLTK Downloader
 
   140     NLTK Downloader
 
   129     ---------------------------------------------------------------------------
 
   141     ---------------------------------------------------------------------------
 
   130         d) Download   l) List    u) Update   c) Config   h) Help   q) Quit
 
   142         d) Download   l) List    u) Update   c) Config   h) Help   q) Quit
 
   131     ---------------------------------------------------------------------------
 
   143     ---------------------------------------------------------------------------
 
   136       - 6 Package Collections Available
 
   148       - 6 Package Collections Available
 
   137       - 107 Individual Packages Available
 
   149       - 107 Individual Packages Available
 
   138 
   150 
   139     Local Machine:
 
   151     Local Machine:
 
   140       - Data directory: /home/tflorac/nltk_data
 
   152       - Data directory: /home/tflorac/nltk_data
 
   141     ---------------------------------------------------------------------------
 
   153 
   142         s) Show Config   u) Set Server URL   d) Set Data Dir   m) Main Menu
 
       
   143     ---------------------------------------------------------------------------
 
       
   144     Config> d
 
   154     Config> d
 
   145       New directory> /usr/local/lib/nltk_data
 
   155       New directory> /usr/local/lib/nltk_data
 
   146 
   156 
       
   157 
       
   158 *4. Return to the main menu:*
 
       
   159 
       
   160 .. code-block:: shell
 
       
   161 
       
   162         ---------------------------------------------------------------------------
 
       
   163             s) Show Config   u) Set Server URL   d) Set Data Dir   m) Main Menu
 
       
   164         ---------------------------------------------------------------------------
 
       
   165         Config> m
 
       
   166 
       
   167 
       
   168 *5. Download utilities:*
 
       
   169 
       
   170     punkt
 
       
   171         Punkt Tokenizer Models
 
       
   172     stopwords
 
       
   173         Stopwords Corpus
 
       
   174 
       
   175 
       
   176 .. code-block:: shell
 
       
   177 
       
   178         ---------------------------------------------------------------------------
 
       
   179             d) Download   l) List    u) Update   c) Config   h) Help   q) Quit
 
       
   180         ---------------------------------------------------------------------------
 
       
   181         Downloader> d
 
       
   182         Download which package (l=list; x=cancel)?
 
       
   183           Identifier> punkt
 
       
   184             Downloading package punkt to /usr/local/lib/nltk_data...
 
       
   185         Downloader> d
 
       
   186         Download which package (l=list; x=cancel)?
 
       
   187           Identifier> stopwords
 
       
   188             Downloading package stopwords to /usr/local/lib/nltk_data...
 
       
   189 
       
   190 
   147 .. tip::
 
   191 .. tip::
 
   148 
   192 
   149     On Debian GNU/Linux, you can choose any directory between '*~/nltk_data*' (where '~' is the homedir of user running
 
   193     The full list of NTLK Collection are displayable with the ``l) list`` option
 
   150     Pyramid application), '*/usr/share/nltk_data*', '*/usr/local/share/nltk_data*', '*/usr/lib/nltk_data*' and
 
       
   151     '*/usr/local/lib/nltk_data*'
 
       
   152 
       
   153 
       
   154 .. code-block:: pycon
 
       
   155 
       
   156     Config> m
 
       
   157     ---------------------------------------------------------------------------
 
       
   158         d) Download   l) List    u) Update   c) Config   h) Help   q) Quit
 
       
   159     ---------------------------------------------------------------------------
 
       
   160     Downloader> d
 
       
   161 
       
   162     Download which package (l=list; x=cancel)?
 
       
   163       Identifier> punkt
 
       
   164         Downloading package punkt to /usr/local/lib/nltk_data...
 
       
   165 
       
   166     Downloader> d
 
       
   167 
       
   168     Download which package (l=list; x=cancel)?
 
       
   169       Identifier> stopwords
 
       
   170         Downloading package stopwords to /usr/local/lib/nltk_data...
pyams/pyams_user_guide