groovy - Creating/updating array of objects in elasticsearch logstash output -


i facing issue using elastic search output logstash. here sample event

{     "guid":"someguid",     "nestedobject":{         "field1":"val1",         "field2":"val2"     } } 

i expect document id present in elasticsearch when update happens. here want have in elastic search document after 2 upserts:

{     "oldfield":"some old field original document before upserts."     "nestedobjects":[{         "field1":"val1",         "field2":"val2"         },         {         "field3":"val3",         "field4":"val4"         }] } 

here current elastic search output setting:

elasticsearch {     index => "elastictest"     action => "update"     document_type => "summary"     document_id => "%{guid}"     doc_as_upsert => true     script_lang => "groovy"     script_type => "inline"     retry_on_conflict => 3     script => "     if (ctx._source.nestedobjects) {     ctx._source.nestedobjects += event.nestedobject     } else {     ctx._source.nestedobjects = [event.nestedobject]     }     "     } 

here error getting:

response=>{"update"=>{"_index"=>"elastictest", "_type"=>"summary", "_id"=>"64648dd3-c1e9-45fd-a00b-5a4332c91ee9", "status"=>400,  "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed parse [event.nestedobject]",  "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"unknown property [field1]"}}}} 

the issue turned out internally generated mapping in elasticsearch due other documents same document_type conflicting type on field nestedobject. caused elastic throw mapper parsing exception. fixing this, fixed issue.


Comments

Popular posts from this blog

How to use SUM() in MySQL for calculated values -

loops - Spock: How to use test data with @Stepwise -