[CMake] TARGET_LINK_LIBRARIES with full path libraries

[Volker Pilipp]

I have encountered the following problem with cmake 3.0.1.
Under certain circumstances TARGET_LINK_LIBRARIES replaces
"/path/to/libXXX.so" by "-lXXX". The problem occurred when I used a
non-standard compiler at /opt/XXX/bin/g++ and added the library
/opt/XXX/lib/libXXX.so to TARGET_LINK_LIBRARIES.
In particular CMakeLists.txt reads:

"SET(CMAKE_CXX_COMPILER /opt/XXX/bin/g++)
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
PROJECT(XXX)
add_executable(xxx xxx.cpp)
target_link_libraries(xxx /opt/XXX/lib/libXXX.so)"

When running make VERBOSE=1 produces output like

/opt/XXX/bin/g++ (...) -lXXX

which is not the same as
/opt/XXX/bin/g++ (...) /opt/XXX/lib/libXXX.so

I am wondering if this is a cmake bug and if there exists a workaround.

[Nils Gladitz]

Post by Volker Pilipp
I have encountered the following problem with cmake 3.0.1.
Under certain circumstances TARGET_LINK_LIBRARIES replaces
"/path/to/libXXX.so" by "-lXXX". The problem occurred when I used a
non-standard compiler at /opt/XXX/bin/g++ and added the library
/opt/XXX/lib/libXXX.so to TARGET_LINK_LIBRARIES.
"SET(CMAKE_CXX_COMPILER /opt/XXX/bin/g++)
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
PROJECT(XXX)
add_executable(xxx xxx.cpp)
target_link_libraries(xxx /opt/XXX/lib/libXXX.so)"
When running make VERBOSE=1 produces output like
/opt/XXX/bin/g++ (...) -lXXX
which is not the same as
/opt/XXX/bin/g++ (...) /opt/XXX/lib/libXXX.so

I think CMake reverts to linking by name if either the library is in an
implicit linker directory (e.g. a directory the linker searches by
default) or if the shared library does not have an SONAME.

Which library does -lXXX resolve to?
e.g. why aren't the two command lines equivalent for you?

Specifically which library does the linker (ld) pick up;
not the runtime loader (ld.so).

Nils

[Volker Pilipp]

Post by Nils Gladitz

Post by Volker Pilipp
I have encountered the following problem with cmake 3.0.1.
Under certain circumstances TARGET_LINK_LIBRARIES replaces
"/path/to/libXXX.so" by "-lXXX". The problem occurred when I used a
non-standard compiler at /opt/XXX/bin/g++ and added the library
/opt/XXX/lib/libXXX.so to TARGET_LINK_LIBRARIES.
"SET(CMAKE_CXX_COMPILER /opt/XXX/bin/g++)
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
PROJECT(XXX)
add_executable(xxx xxx.cpp)
target_link_libraries(xxx /opt/XXX/lib/libXXX.so)"
When running make VERBOSE=1 produces output like
/opt/XXX/bin/g++ (...) -lXXX
which is not the same as
/opt/XXX/bin/g++ (...) /opt/XXX/lib/libXXX.so

I think CMake reverts to linking by name if either the library is in an
implicit linker directory (e.g. a directory the linker searches by default)
or if the shared library does not have an SONAME.
Which library does -lXXX resolve to?
e.g. why aren't the two command lines equivalent for you?

-lXXX stands in the actual case for libcurl.so. I have both
/usr/lib4/libcurl.so.3 and /opt/XXX/llib/libcurl.so.4 installed and I want
my program to be linked against the latter one.

Post by Nils Gladitz
Specifically which library does the linker (ld) pick up;
not the runtime loader (ld.so).

An ldd on the executable produces
libcurl.so.3 => /usr/lib64/libcurl.so.3

Depending on LD_LIBRARY_PATH, (or rpath,...) I would have expected
something like this
libcurl.so.4 => NOT FOUND
or
libcurl.so.4 => /opt/XXX/lib/libcurl.so.4

Volker

[J Decker]

can't just add link_directories( /opt/XXX/lib ) ?

Post by Nils Gladitz

Post by Volker Pilipp
I have encountered the following problem with cmake 3.0.1.
Under certain circumstances TARGET_LINK_LIBRARIES replaces
"/path/to/libXXX.so" by "-lXXX". The problem occurred when I used a
non-standard compiler at /opt/XXX/bin/g++ and added the library
/opt/XXX/lib/libXXX.so to TARGET_LINK_LIBRARIES.
"SET(CMAKE_CXX_COMPILER /opt/XXX/bin/g++)
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
PROJECT(XXX)
add_executable(xxx xxx.cpp)
target_link_libraries(xxx /opt/XXX/lib/libXXX.so)"
When running make VERBOSE=1 produces output like
/opt/XXX/bin/g++ (...) -lXXX
which is not the same as
/opt/XXX/bin/g++ (...) /opt/XXX/lib/libXXX.so

I think CMake reverts to linking by name if either the library is in an
implicit linker directory (e.g. a directory the linker searches by default)
or if the shared library does not have an SONAME.
Which library does -lXXX resolve to?
e.g. why aren't the two command lines equivalent for you?
Specifically which library does the linker (ld) pick up;
not the runtime loader (ld.so).
Nils
--
Powered by www.kitware.com
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at http://www.kitware.com/
opensource/opensource.html
http://public.kitware.com/mailman/listinfo/cmake

[Volker Pilipp]

I did so and found the following strange behaviour

link_directories( /opt/XXX/lib ) -> no effect
but
link_directories( /opt/ ) -> -L/opt (?!)

Volker

Post by J Decker
can't just add link_directories( /opt/XXX/lib ) ?

Post by Nils Gladitz

Post by Volker Pilipp
I have encountered the following problem with cmake 3.0.1.
Under certain circumstances TARGET_LINK_LIBRARIES replaces
"/path/to/libXXX.so" by "-lXXX". The problem occurred when I used a
non-standard compiler at /opt/XXX/bin/g++ and added the library
/opt/XXX/lib/libXXX.so to TARGET_LINK_LIBRARIES.
"SET(CMAKE_CXX_COMPILER /opt/XXX/bin/g++)
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
PROJECT(XXX)
add_executable(xxx xxx.cpp)
target_link_libraries(xxx /opt/XXX/lib/libXXX.so)"
When running make VERBOSE=1 produces output like
/opt/XXX/bin/g++ (...) -lXXX
which is not the same as
/opt/XXX/bin/g++ (...) /opt/XXX/lib/libXXX.so

I think CMake reverts to linking by name if either the library is in an
implicit linker directory (e.g. a directory the linker searches by default)
or if the shared library does not have an SONAME.
Which library does -lXXX resolve to?
e.g. why aren't the two command lines equivalent for you?
Specifically which library does the linker (ld) pick up;
not the runtime loader (ld.so).
Nils
--
Powered by www.kitware.com
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at http://www.kitware.com/
opensource/opensource.html
http://public.kitware.com/mailman/listinfo/cmake

--
**************************************************
Volker Pilipp
Software & IT Technologies

DECTRIS Ltd.
Neuenhoferstr. 107
5400 Baden
Switzerland

[phone] general
[phone] fax
[phone] direct
***@dectris.com
www.dectris.com
**************************************************

Confidentiality Note
This message is intended only for the use of the named recipient(s)
and may contain confidential and/or privileged information. If you are
not the intended recipient, please contact the sender and delete the
message. Any unauthorized use of the information contained in this
message is prohibited.

[Nils Gladitz]

Post by Volker Pilipp
I did so and found the following strange behaviour
link_directories( /opt/XXX/lib ) -> no effect
but
link_directories( /opt/ ) -> -L/opt (?!)

Probably the same logic as for target_link_libraries().
E.g. implicit link directories aren't repeated as explicit link directories.

Nils

[Volker Pilipp]

Post by Nils Gladitz

Post by Volker Pilipp
I did so and found the following strange behaviour
link_directories( /opt/XXX/lib ) -> no effect
but
link_directories( /opt/ ) -> -L/opt (?!)

Probably the same logic as for target_link_libraries().
E.g. implicit link directories aren't repeated as explicit link directories.

But it is not an implicit link directory at least not according to the
output of
"/sbin/ldconfig -p" . What does cmake consider an implicit link directory?

Volker

Post by Nils Gladitz
Nils

--
**************************************************
Volker Pilipp
Software & IT Technologies

DECTRIS Ltd.
Neuenhoferstr. 107
5400 Baden
Switzerland

[phone] general
[phone] fax
[phone] direct
***@dectris.com
www.dectris.com
**************************************************

Confidentiality Note
This message is intended only for the use of the named recipient(s)
and may contain confidential and/or privileged information. If you are
not the intended recipient, please contact the sender and delete the
message. Any unauthorized use of the information contained in this
message is prohibited.

[Nils Gladitz]

Post by Volker Pilipp
I did so and found the following strange behaviour
link_directories( /opt/XXX/lib ) -> no effect
but
link_directories( /opt/ ) -> -L/opt (?!)
Probably the same logic as for target_link_libraries().
E.g. implicit link directories aren't repeated as explicit link directories.
But it is not an implicit link directory at least not according to the
output of
"/sbin/ldconfig -p" . What does cmake consider an implicit link directory?

I think a directory in which the linker looks by default (not the
runtime loader; which is what ldconfig manages).

Run e.g.
echo "int main() {}"|/opt/XXX/bin/g++ -xc++ - -v
And see the directories being passed to collect2 with -L.

Nils

[Volker Pilipp]

Post by Nils Gladitz
Run e.g.
echo "int main() {}"|/opt/XXX/bin/g++ -xc++ - -v

The output is

Target: x86_64-unknown-linux-gnu
Configured with: ../configure --prefix=/opt/XXX
Thread model: posix
gcc version 4.8.2 (GCC)
COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
/opt/XXX/libexec/gcc/x86_64-unknown-linux-gnu/4.8.2/cc1plus -quiet -v
-D_GNU_SOURCE - -quiet -dumpbase - -mtune=generic -march=x86-64 -auxbase -
-version -o /tmp/ccXHzSkQ.s
GNU C++ (GCC) version 4.8.2 (x86_64-unknown-linux-gnu)
compiled by GNU C version 4.8.2, GMP version 4.3.2, MPFR version
2.4.2, MPC version 0.8.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory
"/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../x86_64-unknown-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2/x86_64-unknown-linux-gnu
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2/backward
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/include
/usr/local/include
/opt/XXX/include
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/include-fixed
/usr/include
End of search list.
GNU C++ (GCC) version 4.8.2 (x86_64-unknown-linux-gnu)
compiled by GNU C version 4.8.2, GMP version 4.3.2, MPFR version
2.4.2, MPC version 0.8.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: d8088594c0e624acf8c1ee4300a343b6
COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
as -v --64 -o /tmp/ccytHs8u.o /tmp/ccXHzSkQ.s
GNU assembler version 2.17.50.0.6-26.el5 (x86_64-redhat-linux) using BFD
version 2.17.50.0.6-26.el5 20061020
COMPILER_PATH=/opt/XXX/libexec/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/libexec/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/libexec/gcc/x86_64-unknown-linux-gnu/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/
LIBRARY_PATH=/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
/opt/XXX/libexec/gcc/x86_64-unknown-linux-gnu/4.8.2/collect2
--eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2
/usr/lib/../lib64/crt1.o /usr/lib/../lib64/crti.o
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/crtbegin.o
-L/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2
-L/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64
-L/lib/../lib64 -L/usr/lib/../lib64
-L/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../.. /tmp/ccytHs8u.o
-lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc
/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/crtend.o
/usr/lib/../lib64/crtn.o

I suppose it is the line
LIBRARY_PATH=/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../:/lib/:/usr/lib/

I see that /usr/lib64 (where libcurl.so.3 resides) is before /opt/XXX/lib
(where libcurl.so.4 resides). That's why g++ links against the wrong
library. However, this problem would not occur if I could stop cmake from
replacing /opt/XXX/lib/libcurl.so by -lcurl :-(

Volker

[Nils Gladitz]

Post by Volker Pilipp
I see that /usr/lib64 (where libcurl.so.3 resides) is before
/opt/XXX/lib (where libcurl.so.4 resides). That's why g++ links against
the wrong library. However, this problem would not occur if I could stop
cmake from replacing /opt/XXX/lib/libcurl.so by -lcurl :-(

Maybe you could bring this up on the issue tracker:
http://public.kitware.com/Bug/my_view_page.php

I am not entirely sure what the proper thing to do would be.

Nils

[j s]

Would it be possible to set something like:
TARGET_LINK_LIBRARIES(myexe -L/opt/XXX/lib -lcurl)
in the path?

Post by Nils Gladitz

Post by Volker Pilipp
I see that /usr/lib64 (where libcurl.so.3 resides) is before
/opt/XXX/lib (where libcurl.so.4 resides). That's why g++ links against
the wrong library. However, this problem would not occur if I could stop
cmake from replacing /opt/XXX/lib/libcurl.so by -lcurl :-(

http://public.kitware.com/Bug/my_view_page.php
I am not entirely sure what the proper thing to do would be.
Nils
--
Powered by www.kitware.com
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://public.kitware.com/mailman/listinfo/cmake

[Volker Pilipp]

That actually does the trick. But it's not nice, because there are many
targets in the project.
Actually, I do not know about any use case, where cmake should replace a
full path to a lib
by its "-l" shortcut.

Thank you very much for your help.

Volker

Post by j s
TARGET_LINK_LIBRARIES(myexe -L/opt/XXX/lib -lcurl)
in the path?

Post by Nils Gladitz

Post by Volker Pilipp
I see that /usr/lib64 (where libcurl.so.3 resides) is before
/opt/XXX/lib (where libcurl.so.4 resides). That's why g++ links against
the wrong library. However, this problem would not occur if I could stop
cmake from replacing /opt/XXX/lib/libcurl.so by -lcurl :-(

http://public.kitware.com/Bug/my_view_page.php
I am not entirely sure what the proper thing to do would be.
Nils
--
Powered by www.kitware.com
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://public.kitware.com/mailman/listinfo/cmake

[Nils Gladitz]

Post by Volker Pilipp
That actually does the trick. But it's not nice, because there are many
targets in the project.
Actually, I do not know about any use case, where cmake should replace a
full path to a lib
by its "-l" shortcut.

There are as far as I understand two use cases.

- The library does not have a SONAME
Linking to the library by path would embed the full path of the
library in the target rather than just the name (which is what it does
with -l or when the library does have a SONAME set)

- The second use case I am not familiar with myself but the explanation
in the code is:
// Many system linkers support multiple architectures by
// automatically selecting the implicit linker search path for the
// current architecture. If the library appears in an implicit link
// directory then just report the file name without the directory
// portion. This will allow the system linker to locate the proper
// library for the architecture at link time.

Nils

[J Decker]

It shouldn't truncate paths if specified... cause of course if we want to
test against a dev version then maybe the script knows better. But neither
should it add the full path if it's not specified in the script.

Post by Nils Gladitz

Post by Volker Pilipp
That actually does the trick. But it's not nice, because there are many
targets in the project.
Actually, I do not know about any use case, where cmake should replace a
full path to a lib
by its "-l" shortcut.

There are as far as I understand two use cases.
- The library does not have a SONAME
Linking to the library by path would embed the full path of the library
in the target rather than just the name (which is what it does with -l or
when the library does have a SONAME set)
- The second use case I am not familiar with myself but the explanation in
// Many system linkers support multiple architectures by
// automatically selecting the implicit linker search path for the
// current architecture. If the library appears in an implicit link
// directory then just report the file name without the directory
// portion. This will allow the system linker to locate the proper
// library for the architecture at link time.
Nils
--
Powered by www.kitware.com
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at http://www.kitware.com/
opensource/opensource.html
http://public.kitware.com/mailman/listinfo/cmake

[Xavier Besseron]

Dear all,

I had the same issue some time ago with the Boost library.
The solution for me was to declare this library as "imported".

Here is an example:

add_library( curl UNKNOWN IMPORTED)
set_property(TARGET curl PROPERTY IMPORTED_LOCATION "/opt/XXX/lib/libcurl.so.4")
add_executable(xxx xxx.cpp)
target_link_libraries(xxx curl)


Here are some relevant links:
http://www.cmake.org/pipermail/cmake/2011-June/044790.html
http://www.cmake.org/Wiki/CMake_2.6_Notes#Linking_to_System_Libraries


Regards,

Xavier

Post by J Decker
It shouldn't truncate paths if specified... cause of course if we want to
test against a dev version then maybe the script knows better. But neither
should it add the full path if it's not specified in the script.

Post by Nils Gladitz

Post by Volker Pilipp
That actually does the trick. But it's not nice, because there are many
targets in the project.
Actually, I do not know about any use case, where cmake should replace a
full path to a lib
by its "-l" shortcut.

There are as far as I understand two use cases.
- The library does not have a SONAME
Linking to the library by path would embed the full path of the library
in the target rather than just the name (which is what it does with -l or
when the library does have a SONAME set)
- The second use case I am not familiar with myself but the explanation in
// Many system linkers support multiple architectures by
// automatically selecting the implicit linker search path for the
// current architecture. If the library appears in an implicit link
// directory then just report the file name without the directory
// portion. This will allow the system linker to locate the proper
// library for the architecture at link time.
Nils
--
Powered by www.kitware.com
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://public.kitware.com/mailman/listinfo/cmake

[Alexander Neundorf]

On Wednesday, September 17, 2014 14:50:40 Volker Pilipp wrote:
...

Post by Volker Pilipp
I suppose it is the line
LIBRARY_PATH=/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/lib/g
cc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/:/lib/../lib64/:/usr/lib
/../lib64/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../:/lib/:/
usr/lib/

yes, cmake checks $LIBRARY_PATH and uses -l for the libraries located in these
directories.
I stumbled about that behaviour last year or so, I guess it's a feature.

Alex

[Andreas Naumann]

Post by Alexander Neundorf
...

Post by Volker Pilipp
I suppose it is the line
LIBRARY_PATH=/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/:/opt/XXX/lib/g
cc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/:/lib/../lib64/:/usr/lib
/../lib64/:/opt/XXX/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../:/lib/:/
usr/lib/

yes, cmake checks $LIBRARY_PATH and uses -l for the libraries located in these
directories.
I stumbled about that behaviour last year or so, I guess it's a feature.
Alex

It less a feature of cmake, much more a feature of gcc. It interprets
every directory in LIBRARY_PATH as system directory and reports this to
cmake, so it assumes, that those directories are system directories.

Andreas