Re: a question about the elasticsearch-http driver

hi Fabien,

and thanks for the tip! I have just tried the following destination driver
where the pipeline name is set by the ${HOST} macro:

destination d_elasticsearch_http {
    elasticsearch-http(
        index("syslog-ng")
        type("")
        url("http://localhost:9200/_bulk?pipeline=${HOST}")
        template("$(format-json --scope rfc5424 --scope dot-nv-pairs
        --rekey .* --shift 1 --scope nv-pairs
        --exclude DATE --key ISODATE @timestamp=${ISODATE})")
    );
};

It appears that the macros inside the url are not replaced with their
values. Here is an example request that gets generated:

POST /_bulk?pipeline=${HOST} HTTP/1.1
Host: localhost:9200
User-Agent: syslog-ng 3.32.1/libcurl 7.61.1
Accept: */*
Content-Type: application/x-ndjson
Content-Length: 296
...

So it seems that only constant pipeline names can be used, and one can not
extract the pipeline name from the incoming event.

kind regards,
risto

Kontakt Fabien Wernli (<[email protected]>) kirjutas kuupäeval R, 11. märts
2022 kell 23:11:

> Hi Risto,
>
> Although I didn't test this myself, I see two ways you could achieve this.
>
> 1. By using the "pipeline" query parameter in the url
>    e.g. url("https://localhost:9200/_bulk?pipeline=mypipeline")
> 2. By setting a default pipeline in the ES index
>
> I'm not sure however if elasticsearch-http()'s url parameter is
> templateable. Yoy'll have to check for yourself.
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq