Merge pull request #1 from elasticsearch/master

latest from upstream

Author: pombredanne

.gitignore

@@ -3,6 +3,7 @@
 *.py[co]
 .coverage
 test_elasticsearch/cover
+test_elasticsearch/local.py
 docs/_build
 elasticsearch.egg-info
 .tox

.gitmodules

@@ -1,3 +1,3 @@
-[submodule "test_elasticsearch/rest-api-spec"]
-path = test_elasticsearch/rest-api-spec
-url = https://github.com/elasticsearch/elasticsearch-rest-api-spec.git
+[submodule "rest-api-spec"]
+path = rest-api-spec
+url = https://github.com/elasticsearch/elasticsearch.git

.travis.yml

@@ -3,36 +3,25 @@ language: python
 python:
  - "2.6"
  - "2.7"
+ - "3.2"
  - "3.3"
+ - "3.4"
  - "pypy"
 
 env:
  # different connection classes to test
  - TEST_ES_CONNECTION=Urllib3HttpConnection
  - TEST_ES_CONNECTION=RequestsHttpConnection
- - TEST_ES_CONNECTION=ThriftConnection
-
-matrix:
- exclude:
- # skip testing thrift on py3 and pypy
- - python: "pypy"
- env: TEST_ES_CONNECTION=ThriftConnection
- - python: "3.3"
- env: TEST_ES_CONNECTION=ThriftConnection
 
 install:
- - wget -O - https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.5.tar.gz | tar xz -C /tmp
- - /tmp/elasticsearch-0.90.5/bin/plugin -install elasticsearch/elasticsearch-transport-memcached/1.6.0
- - /tmp/elasticsearch-0.90.5/bin/plugin -install elasticsearch/elasticsearch-transport-thrift/1.6.0
- - git submodule init
- - pip install coveralls
+ - mkdir /tmp/elasticsearch
+ - wget -O - http://s3-us-west-2.amazonaws.com/build.elasticsearch.org/origin/master/nightly/JDK7/elasticsearch-latest-SNAPSHOT.tar.gz | tar xz --directory=/tmp/elasticsearch --strip-components=1
+ - git clone https://github.com/elasticsearch/elasticsearch.git ../elasticsearch
  - pip install .
 
 before_script:
- - /tmp/elasticsearch-0.90.5/bin/elasticsearch -D es.path.data=/tmp -D es.gateway.type=none -D es.index.store.type=memory -D es.discovery.zen.ping.multicast.enabled=false
+ - /tmp/elasticsearch/bin/elasticsearch -d -D es.path.data=/tmp -D es.gateway.type=none -D es.index.store.type=memory -D es.discovery.zen.ping.multicast.enabled=false -Des.node.bench=true
 
 script:
- - TEST_ES_SERVER=localhost python setup.py test
+ - python setup.py test
 
-after_success:
- - coveralls

CONTRIBUTING.md

@@ -20,10 +20,12 @@ The process for contributing to any of the Elasticsearch repositories is similar
 2. Run the test suite to ensure your changes do not break existing code:
 
  ````
- git submodules init
  python setup.py test
  ````
 
+ See the README file in `test_elasticsearch` dirctory for more information on
+ running the test suite.
+
 3. Rebase your changes.
  Update your local repository with the most recent code from the main
  elasticsearch-py repository, and rebase your branch on top of the latest master

Changelog.rst

@@ -3,20 +3,66 @@
 Changelog
 =========
 
-0.4.3
------
+1.2.0 (dev)
+-----------
+
+1.1.0 (2014-07-02)
+------------------
+
+Compatibility with newest Elasticsearch APIs.
+
+ * Test helpers - `ElasticsearchTestCase` and `get_test_client` for use in your
+ tests
+ * Python 3.2 compatibility
+ * Use ``simplejson`` if installed instead of stdlib json library
+ * Introducing a global `request_timeout` parameter for per-call timeout
+ * Bug fixes
+
+1.0.0 (2014-02-11)
+------------------
+
+Elasticsearch 1.0 compatibility. See 0.4.X releases (and 0.4 branch) for code
+compatible with 0.90 elasticsearch.
+
+ * major breaking change - compatible with 1.0 elasticsearch releases only!
+ * Add an option to change the timeout used for sniff requests (`sniff_timeout`).
+ * empty responses from the server are now returned as empty strings instead of None
+ * `get_alias` now has `name` as another optional parameter due to issue #4539
+ in es repo. Note that the order of params have changed so if you are not
+ using keyword arguments this is a breaking change.
+
+0.4.4 (2013-12-23)
+------------------
+
+ * `helpers.bulk_index` renamed to `helpers.bulk` (alias put in place for
+ backwards compatibility, to be removed in future versions)
+ * Added `helpers.streaming_bulk` to consume an iterator and yield results per
+ operation
+ * `helpers.bulk` and `helpers.streaming_bulk` are no longer limitted to just
+ index operations.
+ * unicode body (for `incices.analyze` for example) is now handled correctly
+ * changed `perform_request` on `Connection` classes to return headers as well.
+ This is a backwards incompatible change for people who have developed their own
+ connection class.
+ * changed deserialization mechanics. Users who provided their own serializer
+ that didn't extend `JSONSerializer` need to specify a `mimetype` class
+ attribute.
+ * minor bug fixes
+
+0.4.3 (2013-10-22)
+------------------
 
  * Fixes to `helpers.bulk_index`, better error handling
  * More benevolent `hosts` argument parsing for `Elasticsearch`
  * `requests` no longer required (nor recommended) for install
 
-0.4.2
------
+0.4.2 (2013-10-08)
+------------------
 
  * `ignore` param acceted by all APIs
  * Fixes to `helpers.bulk_index`
 
-0.4.1
------
+0.4.1 (2013-09-24)
+------------------
 
 Initial release.

README.rst

@@ -1,10 +1,42 @@
 Python Elasticsearch Client
 ===========================
 
-Official low-level client for Elasticsearch. It's goal is to provide common
+Official low-level client for Elasticsearch. Its goal is to provide common
 ground for all Elasticsearch-related code in Python; because of this it tries
 to be opinion-free and very extendable.
 
+Compatibility
+-------------
+
+The library is compatible with both Elasticsearch 1.x and 0.90.x but you
+**have to use a matching version**.
+
+For **Elasticsearch 1.0** and later, use the major version 1 (``1.x.y``) of the
+library.
+
+For **Elasticsearch 0.90.x**, use a version from ``0.4.x`` releases of the
+library.
+
+The recommended way to set your requirements in your `setup.py` or
+`requirements.txt` is::
+
+ # Elasticsearch 1.0
+ elasticsearch>=1.0.0,<2.0.0
+
+ # Elasticsearch 0.90
+ elasticsearch<1.0.0
+
+The development is happening on ``master`` and ``0.4`` branches, respectively.
+
+
+Installation
+------------
+
+Install the ``elasticsearch`` package with `pip
+<https://pypi.python.org/pypi/elasticsearch>`_::
+
+ pip install elasticsearch
+
 
 Example use
 -----------
@@ -17,6 +49,10 @@ Simple use-case::
  # by default we connect to localhost:9200
  >>> es = Elasticsearch()
 
+ # create an index in elasticsearch, ignore status code 400 (index already exists)
+ >>> es.indices.create(index='my-index', ignore=400)
+ {u'acknowledged': True}
+
  # datetimes will be serialized
  >>> es.index(index="my-index", doc_type="test-type", id=42, body={"any": "data", "timestamp": datetime.now()})
  {u'_id': u'42', u'_index': u'my-index', u'_type': u'test-type', u'_version': 1, u'ok': True}
@@ -39,7 +75,7 @@ The client's features include:
  decoded for performance reasons)
  * configurable automatic discovery of cluster nodes
  * persistent connections
- * load balancing (with pluggable selection strategy) across all availible nodes
+ * load balancing (with pluggable selection strategy) across all available nodes
  * failed connection penalization (time based - failed connections won't be
  retried until a timeout is reached)
  * thread safety
@@ -69,5 +105,3 @@ Build status
 .. image:: https://secure.travis-ci.org/elasticsearch/elasticsearch-py.png
  :target: http://travis-ci.org/#!/elasticsearch/elasticsearch-py
 
-.. image:: https://coveralls.io/repos/elasticsearch/elasticsearch-py/badge.png?branch=master
- :target: https://coveralls.io/r/elasticsearch/elasticsearch-py

docs/api.rst

@@ -3,13 +3,35 @@
 API Documentation
 =================
 
-.. note::
+All the API calls map the raw REST api as closely as possible, including the
+distinction between required and optional arguments to the calls. This means
+that the code makes distinction between positional and keyword arguments; we,
+however, recommend that people **use keyword arguments for all calls for
+consistency and safety**.
+
+An API call is considered successful (and will return a response) if
+elasticsearch returns a 2XX response. Otherwise an instance of
+:class:`~elasticsearch.TransportError` (or a more specific subclass) will be
+raised. You can see other exception and error states in :ref:`exceptions`. If
+you do not wish an exception to be raised you can always pass in an ``ignore``
+parameter with either a single status code that should be ignored or a list of
+them::
+
+ from elasticsearch import Elasticsearch
+ es = Elasticsearch()
+
+ # ignore 400 cause by IndexAlreadyExistsException when creating an index
+ es.indices.create(index='test-index', ignore=400)
 
- All the API calls map the raw REST api as closely as possible, including
- the distinction between required and optional arguments to the calls. This
- means that the code makes distinction between positional and keyword arguments;
- we, however, recommend that people use keyword arguments for all calls for
- consistency and safety.
+ # ignore 404 and 400
+ es.indices.delete(index='test-index', ignore=[400, 404])
+
+You can also specify ``request_timeout`` (float) as part of any API call, this
+value will get passed to the ``perform_request`` method of the connection
+class::
+
+ # only wait for 1 second, regardless of the client's default
+ es.cluster.health(wait_for_status='yellow', request_timeout=1)
 
 .. note::
 
@@ -38,3 +60,21 @@ Cluster
 .. autoclass:: ClusterClient
  :members:
 
+Nodes
+-----
+
+.. autoclass:: NodesClient
+ :members:
+
+Cat
+---
+
+.. autoclass:: CatClient
+ :members:
+
+Snapshot
+---
+
+.. autoclass:: SnapshotClient
+ :members:
+

docs/conf.py

@@ -50,9 +50,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '0.4.3'
+version = '1.2.0'
 # The full version, including alpha/beta/rc tags.
-release = '0.4.3'
+release = '1.2.0 (dev)'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/exceptions.rst

@@ -0,0 +1,20 @@
+.. _exceptions:
+
+Exceptions
+==========
+
+.. py:module:: elasticsearch
+
+.. autoclass:: ImproperlyConfigured
+
+.. autoclass:: ElasticsearchException
+
+.. autoclass:: SerializationError(ElasticsearchException)
+
+.. autoclass:: TransportError(ElasticsearchException)
+ :members:
+
+.. autoclass:: NotFoundError(TransportError)
+.. autoclass:: ConflictError(TransportError)
+.. autoclass:: RequestError(TransportError)
+.. autoclass:: ConnectionError(TransportError)

docs/helpers.rst

@@ -9,7 +9,9 @@ API.
 
 .. py:module:: elasticsearch.helpers
 
-.. autofunction:: bulk_index
+.. autofunction:: streaming_bulk
+
+.. autofunction:: bulk
 
 .. autofunction:: scan