Re: Kafka-c issues

Sorry, mixed up 18.04 with xenial. Still, it does not make a difference,
Our dbld/build.manifest file has the packaging options for various
platforms and it says:

# on ubuntu, we start using Python3 at focal onwards.
# libcriterion-dev is available starting with 21.04
ubuntu-xenial           python2,nocriterion,nokafka,nojava,nomqtt
ubuntu-bionic           python2,nocriterion,nokafka,nomqtt
ubuntu-focal            python3,nocriterion,nomqtt
ubuntu-impish           python3
ubuntu-jammy            python3

So it'd compile kafka-c with focal onwards. If you have an updated
librdkafka compiled locally, you could remove "nokafka" from the bionic
line, before compiling.

All this assumes that you compile syslog-ng via dbld. If you don't, make
sure that your configure invocation does not disable kafka (e.g. with
--disable-kafka option. by default kafka is auto-detected, so if the dev
package is installed, we would compile with it). All this should be
confirmed by checking the end of the configure output which displays the
modules that are enabled.


On Fri, Jul 15, 2022 at 4:19 PM Sanjay Patel <[email protected]> wrote:

> I have bionic; not sure if that makes a difference?
>
>
>
> *From:* syslog-ng <[email protected]> *On Behalf Of *Balazs
> Scheidler
> *Sent:* Friday, July 15, 2022 3:08 AM
> *To:* Syslog-ng users' and developers' mailing list <
> [email protected]>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> You probably don't have the kafka-c driver installed, this can happen as
> ubuntu 18.04 does not have the required librdkafka libraries. If I remember
> correctly, we don't even supply Xenial packages, but I might be wrong
> there. How did you acquire 3.37.1 for Ubuntu Xenial?
>
>
>
> Our ubuntu-focal packages do have kafka-c, if you check out the build
> logs, it's included:
> https://github.com/syslog-ng/syslog-ng/runs/6776527081?check_suite_focus=true
>
>
>
> ```
>
> Kafka support (module) : yes
>
> ```
>
>
>
> On Fri, Jul 15, 2022 at 8:29 AM Sanjay Patel <[email protected]> wrote:
>
> Attila,
>
> Thanks for the response. Here is the response with your requested change.
>
>
>
> Error parsing config, syntax error, unexpected LL_IDENTIFIER, expecting
> '}' in block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24:4:5-4:12:
>
> 1
>
> 2       #Start Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 3
>
> 4----->     kafka-c(bootstrap_servers("127.0.0.1:9092")
> topic("{incoming}") );
>
> 4----->     ^^^^^^^
>
> 5
>
> 6       #End Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 7
>
>
>
> Included from /etc/syslog-ng/syslog-ng.conf:6:3-9:4:
>
> 1       @version: 3.37
>
> 2       @include "scl.conf"
>
> 3       @define kafka-implementation kafka-c
>
> 4
>
> 5       destination d_kafka {
>
> 6----->   kafka(
>
> 6----->   ^^^^^^^
>
> 7----->     bootstrap-servers("127.0.0.1:9092")
>
> 8----->     topic("{incoming}")
>
> 9----->   );
>
> 10      };
>
>
>
>
>
>
>
> *From:* syslog-ng <[email protected]> *On Behalf Of *Attila
> Szakacs (aszakacs)
> *Sent:* Friday, July 15, 2022 2:25 AM
> *To:* Syslog-ng users' and developers' mailing list <
> [email protected]>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> Hi Sanjay,
>
>
>
> Can you try switching the define and the include, like:
>
>
>
> @version: 3.37
>
> @include "scl.conf"
>
> @define kafka-implementation kafka-c
>
>
>
> ...
>
>
>
> Cheers,
>
> Attila
>
>
> ------------------------------
>
> *From:* syslog-ng <[email protected]> on behalf of
> Sanjay Patel <[email protected]>
> *Sent:* Friday, July 15, 2022 8:07 AM
> *To:* Syslog-ng users' and developers' mailing list <
> [email protected]>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> Here is what happens when I include it. It tires to use kafka-java
>
>
>
> Error parsing config, syntax error, unexpected LL_IDENTIFIER, expecting
> '}' in block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24:4:5-4:15:
>
> 1
>
> 2       #Start Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 3
>
> 4----->     kafka-java(bootstrap_servers("127.0.0.1:9092")
> topic("{incoming}") );
>
> 4----->     ^^^^^^^^^^
>
> 5
>
> 6       #End Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 7
>
>
>
> Included from /etc/syslog-ng/syslog-ng.conf:6:3-9:4:
>
> 1       @version: 3.37
>
> 2       @define kafka-implementation kafka-c
>
> 3       @include "scl.conf"
>
> 4
>
> 5       destination d_kafka {
>
> 6----->   kafka(
>
> 6----->   ^^^^^^^
>
> 7----->     bootstrap-servers("127.0.0.1:9092")
>
> 8----->     topic("{incoming}")
>
> 9----->   );
>
> 10      };
>
>
>
> *From:* syslog-ng <[email protected]> *On Behalf Of *Peter
> Kokai (pkokai)
> *Sent:* Friday, July 15, 2022 2:05 AM
> *To:* Syslog-ng users' and developers' mailing list <
> [email protected]>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> hello,
>
>
>
> the @include is required for kafka, but you commented out with #
>
>
>
> --
>
> kokan
>
>
>
> Get Outlook for Android
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Cattila.szakacs%40balabit.com%7Caaff869a335a430ee9c708da66285200%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637934620622603978%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2Ly%2Fj%2F9kJUQjcC%2Bi3yAjXC6qyevj50Wv5Vx1pUQ9ESU%3D&reserved=0>
> ------------------------------
>
> *From:* syslog-ng <[email protected]> on behalf of
> Sanjay Patel <[email protected]>
> *Sent:* Friday, July 15, 2022 7:30:21 AM
> *To:* [email protected] <[email protected]>
> *Subject:* [syslog-ng] Kafka-c issues
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> I am trying to setup Syslog-NG t send output to Kakka. I am running ubuntu
> 18.04, Syslog-NG version 3.37.
>
>
>
> This is my syslog-ng.conf   ( I have #scl.conf because if I don’t it tries
> to install kafka-java)
>
>
>
>
>
> @version: 3.37
>
> @define kafka-implementation kafka-c
>
> #@include "scl.conf"
>
>
>
> destination d_kafka {
>
>   kafka(
>
>     bootstrap-servers("127.0.0.1:9092")
>
>     topic("{incoming}")
>
>   );
>
> };
>
>
>
>
>
> This is the error when I try and run sudo syslog-ng
>
>
>
> Error parsing config, syntax error, unexpected LL_IDENTIFIER, expecting
> '}' in /etc/syslog-ng/syslog-ng.conf:6:3-6:8:
>
> 1       @version: 3.37
>
> 2       @define kafka-implementation kafka-c
>
> 3       #@include "scl.conf"
>
> 4
>
> 5       destination d_kafka {
>
> 6----->   kafka(
>
> 6----->   ^^^^^
>
> 7           bootstrap-servers("127.0.0.1:9092")
>
> 8           topic("{incoming}")
>
> 9         );
>
> 10      };
>
>
>
> Any help will be appreciated.
>
>
>
> SKP
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
> --
>
> Bazsi
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>

-- 
Bazsi

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq