How to use Elasticsearch aliases with Symfony2 and FOSElastica

by Ben Stinton - 17th April 2015

Symfony2 FOSElastica ElasticSearch Back

Elasticsearch is an incredibly powerful tool which can really help speed up listings and search in your web application. However, as your app grows and your types become more complex it becomes even more important to improve your testing and make further development simpler. Aliases allow you to point different indexes for different environments, and also to seamlessly switch between them.

For this post I'll assume you are using Symfony 2.5.*, FOSElastica bundle 3.0.0alpha6 and Elasticsearch 0.9 to 1.4. For other versions these steps may need checking. I'll also assume that you have setup a test database and bootstrap that works with PHPUnit.

The first step we need to do is create the main indexes, I will use 'index_v1' for prod and dev environments to start with, and 'index_test_v1' for the test environment. i'm going to use the following commands rather than the FOSElastica populate commands:

$ curl -XPUT 'http://localhost:9200/index_v1'
$ curl -XPUT 'http://localhost:9200/index_test_v1'

In your mapping add a line for index_name:

 
# app/config/fos_elastica.yml
fos_elastica:
    clients:
        default: { host: 127.0.0.1, port: 9200 }
    serializer:
         callback_class: FOS\ElasticaBundle\Serializer\Callback
               serializer: serializer
    indexes:
        index:
            index_name: index_%kernel.environment%
            client: default
            types:
                user:

I've unimaginatively called the index, 'index', this doesn't matter and will only be used in the populate commands, the actual index or alias that will be used is defined in 'index_name'. i.e. 'index_%kernel.environment%' points to indexes/aliases 'index_prod', 'index_dev', and 'index_test' depending on the environment.

To create a link from 'index_prod' and 'index_dev' to 'index_v1' use:

$ curl -XPOST 'http://localhost:9200/_aliases' -d '{"actions" : [{ "add" : { "index" : "index_v1", "alias" : "index_prod" } } ]}'
$ curl -XPOST 'http://localhost:9200/_aliases' -d '{"actions" : [{ "add" : { "index" : "index_v1", "alias" : "index_dev" } } ]}'

Finally point 'index_test' to 'index_test_v1'

$ curl -XPOST 'http://localhost:9200/_aliases' -d '{"actions" : [{ "add" : { "index" : "index_test_v1", "alias" : "index_test" } } ]}'

You can check the results with:

$ curl -XGET 'localhost:9200/index_v1/_alias/*'
$ curl -XGET 'localhost:9200/index_test_v1/_alias/*'

Populating the index
N.B. if you use FOSElastica populate command without specifying a type then 'index_v1' will be deleted, and 'index_prod' and 'index_dev' will be created as actual indexes rather than aliases. It also doesn't pick up nested mappings very well, so instead I have populated the types individually:

# for each type defined
$ php app/console fos:elastica:populate --index="index" --type="user" --env="prod" --no-debug

Test Bootstrap
If you are using a bootstrap to initialise/reset your test environment then you need to add the corresponding populate commands after you have recreated your database and loaded any fixtures:

# make sure the environment is set to test
passthru('php "app/console" fos:elastica:populate --index="index" --type="user" --env="test"');

At this point your application should query 'index_v1' for prod and dev environments and 'index_test_v1' for the test environment

Further development
If you want to make further changes to the indexes and types without affecting the prod environment index, and so you can easily roll back any changes, then you can create new indexes

$ curl -XPUT 'http://localhost:9200/index_v2'
$ curl -XPUT 'http://localhost:9200/index_test_v2'

You can then switch the 'index_dev' alias to 'index_v2' with the following command:

$ curl -XPOST 'http://localhost:9200/_aliases' -d '{ "actions" : [ { "remove" : { "index" : "index_v1", "alias" : "index_dev" } }, { "add" : { "index" : "index_v2", "alias" : "index_dev" } } ] }'

At this point you can do the same with the test environment.

$ curl -XPOST 'http://localhost:9200/_aliases' -d '{ "actions" : [ { "remove" : { "index" : "index_test_v1", "alias" : "index_test" } }, { "add" : { "index" : "index_test_v2", "alias" : "index_test" } } ] }'

Once you are happy with you changes you can use the dev environment to populate your new 'index_v2' and then seamlessly switch the production environment over as well. Finally you can delete 'index_v1' when you are happy it is no longer needed.

$ curl -XDELETE 'http://localhost:9200/index_v1'

N.B. Remember that any entity updates or inserts made in the dev environment while it is pointing at 'index_v2' will obviously not be reflected in 'index_v1'.

Links:
https://www.elastic.co/blog/changing-mapping-with-zero-downtime/