"Undefined symbols for x86_64" link error for Kafka C++ in MacOS

  apache-kafka, c++, librdkafka, macos

I’ve been trying to build a simple consumer for Kafka in C++ using VsCode in MacOS (Catalina version 10.15.7) but I hit the same linkage error. I tried to find some answers around here, but the ones I found are not useful for me (as this one, for example) because I can build Kafka properly.

Here are the steps I tried up to now and the outputs of the compiler/linker in each case. I didn’t include the code as it is a simple code and the errors come from the linker, although I will ad it if someone thinks it could help. Let me know if I should include anything else.

Installing with brew

First I installed kafka libraries using brew with the following command:
brew install librdkafka
After installation, the libraries were located in /usr/local/Cellar/librdkafka/1.7.0/lib/ and /usr/local/Cellar/librdkafka/1.7.0/include/, which were the paths I used in task.json for the default build in VsCode.

The error I got was this:

Undefined symbols for architecture x86_64:
  "RdKafka::get_debug_contexts()", referenced from:
      _main in read-topics-d64669.o
  "RdKafka::Conf::create(RdKafka::Conf::ConfType)", referenced from:
      _main in read-topics-d64669.o
  "RdKafka::Topic::OFFSET_BEGINNING", referenced from:
      _main in read-topics-d64669.o
  "RdKafka::Consumer::create(RdKafka::Conf const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)", referenced from:
      _main in read-topics-d64669.o
...

(shortened for clarity).
If I use IntelliSense to look for the definition of any Kafka function used in my code, it finds it properly opening the library file in the correct folder ('/usr/local/Cellar/librdkafka/1.7.0/include/librdkafka/rdkafkacpp.h' ) as added in tasks.json (Reference 1). However the linker seems to have problems finding the library file that is also added to the linker parameters.

Compiling from sources

Then I tried to compile Kafka directly from sources following the instructions provided in the repository’s documentation. To be sure I was not missing anything, I added the option that automatically installs the dependencies needed: ./configure --install-deps.

The only suspicious message from that command was this one

WARNING: librdkafka-static.a: Not creating self-contained static library librdkafka-static.a: no static libraries available/enabled

but everything looked good and I don’t think that’s the real issue. Them, I installed them with make install and the output is available below (Ref 3).

Then I modified tasks.json to have the correct path to both include and lib folders (Reference 2). However, building still fails due to the linker and I have the same error as before:

Undefined symbols for architecture x86_64:
  "RdKafka::get_debug_contexts()", referenced from:
      _main in read-topics-991879.o
  "RdKafka::Conf::create(RdKafka::Conf::ConfType)", referenced from:
      _main in read-topics-991879.o
  "RdKafka::Topic::OFFSET_BEGINNING", referenced from:
      _main in read-topics-991879.o
 ...

Any idea of what I could try to solve the error?
What I am doing wrong or what am I missing?
Any help or suggestion will be appreciated.


References

  1. tasks.json with Kafka installed with brew
{
    "version": "2.0.0",
    "tasks": [
        {
            "type": "shell",
            "label": "C/C++: clang build active file",
            "command": "/usr/bin/clang",
            "args": [
                "-std=c++17",
                "-stdlib=libc++",
                "-g",
                "-v",
                "-I/usr/local/Cellar/librdkafka/1.7.0/include/librdkafka",
                "-L/usr/local/Cellar/librdkafka/1.7.0/lib/",
                "${file}",
                "-o",
                "${fileDirname}/${fileBasenameNoExtension}"
            ],
            "options": {
                "cwd": "${workspaceFolder}"
            },
            "problemMatcher": [
                "$gcc"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}
  1. tasks.json with Kafka compiled from source (only the part that changed, everything else is the same)
...
"args": [
                "-std=c++17",
                "-stdlib=libc++",
                "-g",
                "-v",
                "-I/usr/local/include/librdkafka/",
                "-L/usr/local/lib",
                "${file}",
                "-o",
                "${fileDirname}/${fileBasenameNoExtension}"
            ],
...
  1. Kafka Build Output
Install librdkafka to /usr/local
install -d $DESTDIR/usr/local/include/librdkafka
install -d $DESTDIR/usr/local/lib
install rdkafka.h rdkafka_mock.h $DESTDIR/usr/local/include/librdkafka
install librdkafka.a $DESTDIR/usr/local/lib
[ ! -f librdkafka-static.a ] || install librdkafka-static.a $DESTDIR/usr/local/lib
install librdkafka.1.dylib $DESTDIR/usr/local/lib
[ -f "rdkafka.pc" ] && ( 
        install -d $DESTDIR/usr/local/lib/pkgconfig && 
        install -m 0644 rdkafka.pc $DESTDIR/usr/local/lib/pkgconfig 
    )
[ -f "rdkafka-static.pc" ] && ( 
        install -d $DESTDIR/usr/local/lib/pkgconfig && 
        install -m 0644 rdkafka-static.pc $DESTDIR/usr/local/lib/pkgconfig 
    )
(cd $DESTDIR/usr/local/lib && ln -sf librdkafka.1.dylib librdkafka.dylib)
Install librdkafka++ to /usr/local
install -d $DESTDIR/usr/local/include/librdkafka
install -d $DESTDIR/usr/local/lib
install rdkafkacpp.h $DESTDIR/usr/local/include/librdkafka
install librdkafka++.a $DESTDIR/usr/local/lib
[ ! -f librdkafka++-static.a ] || install librdkafka++-static.a $DESTDIR/usr/local/lib
install librdkafka++.1.dylib $DESTDIR/usr/local/lib
[ -f "rdkafka++.pc" ] && ( 
        install -d $DESTDIR/usr/local/lib/pkgconfig && 
        install -m 0644 rdkafka++.pc $DESTDIR/usr/local/lib/pkgconfig 
    )
[ -f "rdkafka++-static.pc" ] && ( 
        install -d $DESTDIR/usr/local/lib/pkgconfig && 
        install -m 0644 rdkafka++-static.pc $DESTDIR/usr/local/lib/pkgconfig 
    )
(cd $DESTDIR/usr/local/lib && ln -sf librdkafka++.1.dylib librdkafka++.dylib)
WARNING: librdkafka-static.a: Not creating self-contained static library librdkafka-static.a: no static libraries available/enabled
Generating pkg-config file rdkafka-static.pc
Checking librdkafka integrity
librdkafka.1.dylib             OK
librdkafka.a                   OK
Symbol visibility              OK
Generating pkg-config file rdkafka++-static.pc
Checking librdkafka++ integrity
librdkafka++.1.dylib           OK
librdkafka++.a                 OK
/Library/Developer/CommandLineTools/usr/bin/make -C examples
make[1]: Nothing to be done for `all'.
Updating CONFIGURATION.md
Installing documentation to /usr/local
install -d $DESTDIR/usr/local/share/doc/librdkafka
install LICENSE LICENSES.txt INTRODUCTION.md README.md CONFIGURATION.md STATISTICS.md CHANGELOG.md $DESTDIR/usr/local/share/doc/librdkafka

Source: Windows Questions C++

LEAVE A COMMENT