Elastic Search synonyms

[justbane]

Hello all, I am incredibly stuck.

I have elastic search working on the demo install (Car Shop)

I am trying to show an example to a client of synonyms working but I can not make it work with the provided documentation.

Here is the portion of my elastic-search.yml file

                    client_config:
                        logging: true
                        indexName: 'pimcore-demo'

                    es_client_params:
                        hosts:
                            - 'elasticserch'
                        indexType: 'DemoProducts'

                    synonym_providers:
                        app_synonym_filter:
                            # service ID of synonym provider to use
                            provider_id: Pimcore\Bundle\EcommerceFrameworkBundle\IndexService\SynonymProvider\FileSynonymProvider
                            # additional options for synonym provider (are applied to a child-service instance of given synonym provider
                            options:
                                synonymFile: '%kernel.project_dir%/web/var/assets/synonyms.txt'

                    index_settings:
                        number_of_shards: 5
                        number_of_replicas: 0
                        analysis:
                            normalizer:
                                lowercase:
                                    type: custom
                                    filter:
                                        - lowercase
                            analyzer:
                                app_ngram_analyzer:
                                    tokenizer: app_ngram_tokenzier
                                    filter:
                                        - lowercase
                                app_whitespace_analyzer:
                                    tokenizer: app_whitespace_tokenzier
                                    filter:
                                        - lowercase
                            tokenizer:
                                app_ngram_tokenzier:
                                    type: nGram
                                    min_gram: 3
                                    max_gram: 25
                                    token_chars: [letter, digit]
                                app_whitespace_tokenzier:
                                    type: whitespace
                            filter:
                                app_synonym_filter: # mapping between synonym_provider and filter based on name
                                    type: synonym # or synonym_graph or any other ES type
                                    synonyms: [] # provide an empy array here, for ES6 [], for ES5[""]

This is from the demo and I added the snippets from the documentation for Synonyms here: Synonyms

If I index the products with this.

bin/console ecommerce:indexservice:bootstrap --update-index --object-list-class="\Pimcore\Model\DataObject\Car\Listing" --tenant="ElasticSearch"

Everything goes fine no errors - but of course you can not search anything using synonyms

If I search using:

bin/console ecommerce:indexservice:elasticsearch-sync update-synonyms --tenant="ElasticSearch"

I get this error...

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"filter [app_synonym_filter] must specify either an analyzer type, or a tokenizer"}],"type":"illegal_argument_exception","rea  
  son":"filter [app_synonym_filter] must specify either an analyzer type, or a tokenizer"},"status":400}

ANY HELP would be appreciated - even a link that spells it out would be great.

[fashxp]

Hi,
you need to add the synonym filter also to an analyser ... I added a note to the docs, see eb6ce26

[justbane]

Thank you @fashxp

I tried that and it still through the error

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"filter [app_synonym_filter] must specify either an analyzer type, or a tokenizer"}],"type":"illegal_argument_exception","rea  
  son":"filter [app_synonym_filter] must specify either an analyzer type, or a tokenizer"},"status":400}

What I tried then was to add a tokenizer to the actual filter definition, like so...

filter:
    app_synonym_filter: # mapping between synonym_provider and filter based on name
        type: synonym # or synonym_graph or any other ES type
        synonyms: [] # provide an empy array here, for ES6 [], for ES5[""]
        tokenizer: standard

This cleared the error and added the synonyms to the index. Obviously you do need to add the filter to an analyzer to USE the synonyms but to add them to the index you need to have the tokenizer in the filter definition it seems.

Not sure if these needs verification or not or who might do that.

[fashxp]

Intereseting, for me it was enough to just add filter to a filter list of an analyser, like so

analyzer:
    app_ngram_analyzer:
        tokenizer: app_ngram_tokenzier
        filter:
            - lowercase
            - app_synonym_filter

well, as long as it works now...