[CMake] ExternalProjects: How do ExternalProject_add and ExternalProject_Add_Step interact?

[kent williams]

So... I have been working on my 'Holy Grail' CMakeLists.txt that
builds all prerequisites to my application as ExternalProjects, and
then builds my application (also as an External Project).

In getting it to work on OS X and Linux I ran into an issue:

If you build Tcl, and install it, it installs the shared libraries for
Tcl read-only. So, if I try and use fixup_bundle (from
BundleUtilities) to make a standalone build with all shared libraries
it fails:

-- 82/204: copying '/scratch/kent/newbuild/BT-build/lib/libtk8.5.so'
CMake Error at /opt/cmake-2.9.20100421/share/cmake-2.9/Modules/BundleUtilities.cmake:431
(file):
file RPATH_REMOVE could not remove RPATH from file:

/scratch/kent/newbuild/BT-build/bin/libtk8.5.so

Error opening file for update.
Call Stack (most recent call first):
/opt/cmake-2.9.20100421/share/cmake-2.9/Modules/BundleUtilities.cmake:531
(copy_resolved_item_into_bundle)
cmake_install.cmake:54 (fixup_bundle)

So my thought was 'brilliant! I'll add a step to the ExternalProject!'
-- but it isn't clear to me if ExternalProject_Add_Step is actually
set up to modify an External Project, or if you're supposed to use
ExternalProject_Add_Step commands to swizzle up your own custom
External Project.

More specifically If I have

ExternalProjectAdd(xyz
...
)

ExternalProject_Add_Step(xyz fix_permissions
...
)

I don't know what to put for DEPENDEES to get fix_permissions to run
AFTER the install.

Any ideas?

[kent williams]

Answering my own question -- I think. This maybe is information that
could be added to the documentation for ExternalProjects.cmake!

ExternalProject_Add creates a project, e.g.

ExternalProject_Add(tcl
CVS_REPOSITORY ":pserver:anonymous:@tcl.cvs.sourceforge.net:/cvsroot/tcl"
CVS_TAG -r core-8-5-7
CVS_MODULE "tcl"
CONFIGURE_COMMAND
${BRAINSTracer_BINARY_DIR}/tcl-prefix/src/tcl/unix/configure
--prefix=${CMAKE_BINARY_DIR}
INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}
UPDATE_COMMAND ""
)

When the generated makefiles actually build the project, there's a
directory named

${CMAKE_CURRENT_BINARY_DIR}/tcl-prefix/src/tcl-stamp

In which a 0-length file is created after each step of the external
project build is successfully finished. These files (and THIS is what
was undocumented, that I had to determine by poking around in the
build directories) are named

${proj}-mkdir ${proj}-download ${proj}-update ${proj}-patch
${proj}-configure ${proj}-build ${proj}-install ${proj}-done

Where 'proj' is the name you've given the ExternalProject. So in
order to add a step you have to list the steps your new custom step
depends on, after the DEPENDEES keyword. For example:

if(APPLE)
set(SHARED_LIB_EXT .dylib)
else(APPLE)
set(SHARED_LIB_EXT .so)
endif(APPLE)

ExternalProject_Add_Step(${proj} after_install
COMMAND chmod u+w ${BRAINSTracer_BINARY_DIR}/lib/libtcl8.5${SHARED_LIB_EXT}
COMMENT "----------------------
${BRAINSTracer_BINARY_DIR}/lib/libtcl8.5.so installed read-only!"
DEPENDEES mkdir update patch download configure build install
)

Now it appears that you have to list ALL the prerequisite steps, not
just the one after which your step should occur.

[Luigi Calori]

There is some info at
http://www.kitware.com/products/archive/kitware_quarterly1009.pdf

based on that I thought it was enouth do depend upon just the install in
your case...

I am collecting external projects for several libs, It seem you are
building at least VTK and Tcl and Tk that I lack...
Do you mind sharing?

my stuff is (dis)organized as a series of packages and assemblies... far
for being cross platform and complete though
If you like, have a look at
http://3d.cineca.it/storage/bazaar_repo/CmakeDeps/lib/wt/

Thanks in advance

Luigi

Post by kent williams
Answering my own question -- I think. This maybe is information that
could be added to the documentation for ExternalProjects.cmake!
ExternalProject_Add creates a project, e.g.
ExternalProject_Add(tcl
CVS_TAG -r core-8-5-7
CVS_MODULE "tcl"
CONFIGURE_COMMAND
${BRAINSTracer_BINARY_DIR}/tcl-prefix/src/tcl/unix/configure
--prefix=${CMAKE_BINARY_DIR}
INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}
UPDATE_COMMAND ""
)
When the generated makefiles actually build the project, there's a
directory named
${CMAKE_CURRENT_BINARY_DIR}/tcl-prefix/src/tcl-stamp
In which a 0-length file is created after each step of the external
project build is successfully finished. These files (and THIS is what
was undocumented, that I had to determine by poking around in the
build directories) are named
${proj}-mkdir ${proj}-download ${proj}-update ${proj}-patch
${proj}-configure ${proj}-build ${proj}-install ${proj}-done
Where 'proj' is the name you've given the ExternalProject. So in
order to add a step you have to list the steps your new custom step
if(APPLE)
set(SHARED_LIB_EXT .dylib)
else(APPLE)
set(SHARED_LIB_EXT .so)
endif(APPLE)
ExternalProject_Add_Step(${proj} after_install
COMMAND chmod u+w ${BRAINSTracer_BINARY_DIR}/lib/libtcl8.5${SHARED_LIB_EXT}
COMMENT "----------------------
${BRAINSTracer_BINARY_DIR}/lib/libtcl8.5.so installed read-only!"
DEPENDEES mkdir update patch download configure build install
)
Now it appears that you have to list ALL the prerequisite steps, not
just the one after which your step should occur.
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
http://www.cmake.org/mailman/listinfo/cmake

[David Cole]

Post by Luigi Calori
There is some info at
http://www.kitware.com/products/archive/kitware_quarterly1009.pdf
based on that I thought it was enouth do depend upon just the install in
your case...

Luigi is correct: it should be sufficient to list "DEPENDEES install" in
your case. Kent, why do you think you have to list all the steps as
DEPENDEES?

[kent williams]

I tried just listing the single step, and for some reason (probably my
fault) it didn't work.

I did get this working, but I had to add two things to keep tcl and tk
from breaking fixup_bundle -- I had to make the Tcl/Tk SO libraries
writable, and I needed to delete tclsh and wish, because fixup_bundle
didn't like their link dependencies.

set(TCL_DEPEND tcl)
set(proj tcl)
ExternalProject_Add(${proj}
CVS_REPOSITORY ":pserver:anonymous:@tcl.cvs.sourceforge.net:/cvsroot/tcl"
CVS_TAG -r core-8-5-7
CVS_MODULE "tcl"
CONFIGURE_COMMAND
sh ${CONFIG_WRAPPER}
${BRAINSTracer_BINARY_DIR}/tcl-prefix/src/tcl/unix/configure
--prefix=${BRAINSTracer_BINARY_DIR}
INSTALL_DIR ${BRAINSTracer_BINARY_DIR}
UPDATE_COMMAND ""
)

if(APPLE)
set(SHARED_LIB_EXT .dylib)
else(APPLE)
set(SHARED_LIB_EXT .so)
endif(APPLE)

ExternalProject_Add_Step(${proj} after_install
COMMAND chmod u+w ${BRAINSTracer_BINARY_DIR}/lib/libtcl8.5${SHARED_LIB_EXT}
COMMAND cmake -E remove ${BRAINSTracer_BINARY_DIR}/bin/tclsh8.5
COMMENT "----------------------
${BRAINSTracer_BINARY_DIR}/lib/libtcl8.5 installed read-only!"
DEPENDEES mkdir update patch download configure build install
)

Post by David Cole

Post by Luigi Calori
There is some info at
http://www.kitware.com/products/archive/kitware_quarterly1009.pdf
based on that I thought it was enouth do depend upon just the install in
your case...

Luigi is correct: it should be sufficient to list "DEPENDEES install" in
your case. Kent, why do you think you have to list all the steps as
DEPENDEES?