[CMake] Running cmake multiple times to resolve cache variables

[Luke Kucalaba]

I have a question about knowing when you have run cmake enough times to
resolve all the cache variables for a project. In the CMake explanation
on the main website, regarding generation using the cmake.exe program
from the command-line in non-interactive mode, it says "It can be
difficult to know when to stop the run cmake, edit the cache file cycle
without the aid of an interface." Yet when you run the interactive
version of cmake (CMakeSetup or ccmake), the program detects when you
have resolved all cache variables (for example, CMakeSetup uses
color-coding, red=not resolved, gray=resolved), and will not even let
you generate projects/makefiles until all cache variables are resolved
(grayed). Why hasn't this detection capability been added to cmake.exe
for non-interactive mode? And why does the cmake.exe in non-interactive
mode generate "intermediate" build files that are not "fully generated"
because you haven't yet resolved all the cache variables, yet the
interactive versions don't even generate build files until the very
final step?



Currently, the output always looks like this:

-- Configuring done

-- Generating done

-- Build files have been written to:
N:/dsci/repo/WMI/tests/CMakeTest/msvc71



How difficult would it be to put a warning message like "X cache
variables are still unresolved. Running cmake again may resolve these
variables."



-- Configuring done

-- Generating done

-- WARNING: 123 cache variables are still unresolved, recommend running
CMake again to resolve



Does anyone else think this would be a useful feature to add? Should I
make a feature request in Mantis?



Luke

[Bill Hoffman]

Post by Luke Kucalaba
I have a question about knowing when you have run cmake enough times to
resolve all the cache variables for a project. In the CMake explanation
on the main website, regarding generation using the cmake.exe program
from the command-line in non-interactive mode, it says “It can be
difficult to know when to stop the run cmake, edit the cache file cycle
without the aid of an interface.” Yet when you run the interactive
version of cmake (CMakeSetup or ccmake), the program detects when you
have resolved all cache variables (for example, CMakeSetup uses
color-coding, red=not resolved, gray=resolved), and will not even let
you generate projects/makefiles until all cache variables are resolved
(grayed). Why hasn’t this detection capability been added to cmake.exe
for non-interactive mode? And why does the cmake.exe in non-interactive
mode generate “intermediate” build files that are not “fully generated”
because you haven’t yet resolved all the cache variables, yet the
interactive versions don’t even generate build files until the very
final step?

Not sure this makes sense. The new variables show up in the cache with
a default value. Since you are running from the command line there is
no way to change the default value, so there is no reason to run cmake
again (nothing is going to change.). With the gui's the user is
presented with new cache values and given the chance to change them.

-Bill

[Luke Kucalaba]

I think the reason we need to run cmake.exe multiple times is because we
typically have a "main" project that automatically sets the options of
dependent projects. For example, our main executable project has an
option "ENABLE_NETWORKING", which then triggers automatic changes for
various global CACHE variables that are used in subdirectories
(sub-projects) that have already been processed (using CACHE STRING
FORCE). This requires the user to run cmake.exe again to reprocess
those subdirectories. Is this not something that we should be doing?
Or am I not understanding how the multiple passes of cmake works?

Luke

-----Original Message-----
From: Bill Hoffman [mailto:***@kitware.com]
Sent: Friday, October 10, 2008 11:12 AM
To: Luke Kucalaba
Cc: ***@cmake.org
Subject: Re: [CMake] Running cmake multiple times to resolve cache
variables

Post by Luke Kucalaba
I have a question about knowing when you have run cmake enough times to
resolve all the cache variables for a project. In the CMake

explanation

Post by Luke Kucalaba
on the main website, regarding generation using the cmake.exe program
from the command-line in non-interactive mode, it says "It can be
difficult to know when to stop the run cmake, edit the cache file cycle
without the aid of an interface." Yet when you run the interactive
version of cmake (CMakeSetup or ccmake), the program detects when you
have resolved all cache variables (for example, CMakeSetup uses
color-coding, red=not resolved, gray=resolved), and will not even let
you generate projects/makefiles until all cache variables are resolved
(grayed). Why hasn't this detection capability been added to

cmake.exe

Post by Luke Kucalaba
for non-interactive mode? And why does the cmake.exe in

non-interactive

Post by Luke Kucalaba
mode generate "intermediate" build files that are not "fully

generated"

Post by Luke Kucalaba
because you haven't yet resolved all the cache variables, yet the
interactive versions don't even generate build files until the very
final step?

Not sure this makes sense. The new variables show up in the cache with

a default value. Since you are running from the command line there is
no way to change the default value, so there is no reason to run cmake
again (nothing is going to change.). With the gui's the user is
presented with new cache values and given the chance to change them.

-Bill

[Bill Hoffman]

Post by Luke Kucalaba
I think the reason we need to run cmake.exe multiple times is because we
typically have a "main" project that automatically sets the options of
dependent projects. For example, our main executable project has an
option "ENABLE_NETWORKING", which then triggers automatic changes for
various global CACHE variables that are used in subdirectories
(sub-projects) that have already been processed (using CACHE STRING
FORCE). This requires the user to run cmake.exe again to reprocess
those subdirectories. Is this not something that we should be doing?
Or am I not understanding how the multiple passes of cmake works?
Luke

I still don't get your problem. The use of CACHE FORCE is usually a bad
idea. A CMake project should not need to be run twice to work
correctly. Basically, you have created cmake files that do not use the
correct order of evaluation.

Here is one example of that:

add_library(foo ...)
# I use SOME_LIB before it is set
target_link_library(foo ${SOME_LIB})

# Now I add a sub directory A, that sets SOME_LIB as a cache variable
add_subdirectory(A)
A: set(SOME_LIB mylib CACHE FORCE)

This would require two runs of cmake to get the value of SOME_LIB
correct in the toplevel cmake file.

Basically, if your project requires double runs of CMake, then it has a bug.

-Bill

[Luke Kucalaba]

Thanks Bill, I understand exactly what you are talking about. I looked
into our cmake project structure, and ran some tests. I think we
originally started out using a sequentially correct organization, but we
ran into problems with the Include_Directories() command passing down
the include directories to each subdirectory, when we really want a
subdirectory to be treated as a completely stand-alone module (for
instance, an independent static library project). The other problem is
that the CMAKE_CONFIGURATION_TYPES variable doesn't seem to get applied
unless you run cmake a second time (we only want to have Debug and
Release configurations in MSVC71).

You are completely correct in that we needed to reorganize our project
structure. I came up with a workaround for the include directories
problem.

This was our old sequence (required two passes):

Add_Subdirectory(LIB) #load library project
Set(EXE_DEF "somedef") #executable defs
Set(LIB_DEF "somedef") #library def overrides
Include_Directories() #set include directories for exe
Add_Executable(EXE)
Target_Link_Libraries(EXE LIB)
Add_Dependencies(EXE LIB)

Here is our sequence now (only needs one pass):

Set(EXE_DEF "somedef") #executable defs
Set(LIB_DEF "somedef") #library def overrides
Add_Subdirectory(LIB) #load library project
Include_Directories() #set include directories for exe
Add_Executable(EXE)
Target_Link_Libraries(EXE LIB)
Add_Dependencies(EXE LIB)

By putting the Include_Directories() after the Add_Subdirectory() it
prevented the unwanted behavior.

Still don't know how to set the build configurations with
CMAKE_CONFIGURATION_TYPES using a single pass, but other than that this
seems to be working with one pass. Thanks for your help :-D

Luke


-----Original Message-----
From: Bill Hoffman [mailto:***@kitware.com]
Sent: Friday, October 10, 2008 12:42 PM
To: Luke Kucalaba
Cc: ***@cmake.org
Subject: Re: [CMake] Running cmake multiple times to resolve cache
variables

Post by Luke Kucalaba
I think the reason we need to run cmake.exe multiple times is because we
typically have a "main" project that automatically sets the options of
dependent projects. For example, our main executable project has an
option "ENABLE_NETWORKING", which then triggers automatic changes for
various global CACHE variables that are used in subdirectories
(sub-projects) that have already been processed (using CACHE STRING
FORCE). This requires the user to run cmake.exe again to reprocess
those subdirectories. Is this not something that we should be doing?
Or am I not understanding how the multiple passes of cmake works?
Luke

I still don't get your problem. The use of CACHE FORCE is usually a bad

idea. A CMake project should not need to be run twice to work
correctly. Basically, you have created cmake files that do not use the
correct order of evaluation.

Here is one example of that:

add_library(foo ...)
# I use SOME_LIB before it is set
target_link_library(foo ${SOME_LIB})

# Now I add a sub directory A, that sets SOME_LIB as a cache variable
add_subdirectory(A)
A: set(SOME_LIB mylib CACHE FORCE)

This would require two runs of cmake to get the value of SOME_LIB
correct in the toplevel cmake file.

Basically, if your project requires double runs of CMake, then it has a
bug.

-Bill

[Luke Kucalaba]

Correction, that email somehow got corrupted wrt the newlines. These
should be on two separate lines:

Set(EXE_DEF "somedef") #executable defs
Set(LIB_DEF "somedef") #library def overrides

Sorry about that,
Luke

-----Original Message-----
From: Luke Kucalaba
Sent: Friday, October 10, 2008 1:46 PM
To: 'Bill Hoffman'
Cc: ***@cmake.org
Subject: RE: [CMake] Running cmake multiple times to resolve cache
variables

Thanks Bill, I understand exactly what you are talking about. I looked
into our cmake project structure, and ran some tests. I think we
originally started out using a sequentially correct organization, but we
ran into problems with the Include_Directories() command passing down
the include directories to each subdirectory, when we really want a
subdirectory to be treated as a completely stand-alone module (for
instance, an independent static library project). The other problem is
that the CMAKE_CONFIGURATION_TYPES variable doesn't seem to get applied
unless you run cmake a second time (we only want to have Debug and
Release configurations in MSVC71).

You are completely correct in that we needed to reorganize our project
structure. I came up with a workaround for the include directories
problem.

This was our old sequence (required two passes):

Add_Subdirectory(LIB) #load library project
Set(EXE_DEF "somedef") #executable defs
Set(LIB_DEF "somedef") #library def overrides
Include_Directories() #set include directories for exe
Add_Executable(EXE)
Target_Link_Libraries(EXE LIB)
Add_Dependencies(EXE LIB)

Here is our sequence now (only needs one pass):

Set(EXE_DEF "somedef") #executable defs
Set(LIB_DEF "somedef") #library def overrides
Add_Subdirectory(LIB) #load library project
Include_Directories() #set include directories for exe
Add_Executable(EXE)
Target_Link_Libraries(EXE LIB)
Add_Dependencies(EXE LIB)

By putting the Include_Directories() after the Add_Subdirectory() it
prevented the unwanted behavior.

Still don't know how to set the build configurations with
CMAKE_CONFIGURATION_TYPES using a single pass, but other than that this
seems to be working with one pass. Thanks for your help :-D

Luke


-----Original Message-----
From: Bill Hoffman [mailto:***@kitware.com]
Sent: Friday, October 10, 2008 12:42 PM
To: Luke Kucalaba
Cc: ***@cmake.org
Subject: Re: [CMake] Running cmake multiple times to resolve cache
variables

Post by Luke Kucalaba
I think the reason we need to run cmake.exe multiple times is because we
typically have a "main" project that automatically sets the options of
dependent projects. For example, our main executable project has an
option "ENABLE_NETWORKING", which then triggers automatic changes for
various global CACHE variables that are used in subdirectories
(sub-projects) that have already been processed (using CACHE STRING
FORCE). This requires the user to run cmake.exe again to reprocess
those subdirectories. Is this not something that we should be doing?
Or am I not understanding how the multiple passes of cmake works?
Luke

I still don't get your problem. The use of CACHE FORCE is usually a bad

idea. A CMake project should not need to be run twice to work
correctly. Basically, you have created cmake files that do not use the
correct order of evaluation.

Here is one example of that:

add_library(foo ...)
# I use SOME_LIB before it is set
target_link_library(foo ${SOME_LIB})

# Now I add a sub directory A, that sets SOME_LIB as a cache variable
add_subdirectory(A)
A: set(SOME_LIB mylib CACHE FORCE)

This would require two runs of cmake to get the value of SOME_LIB
correct in the toplevel cmake file.

Basically, if your project requires double runs of CMake, then it has a
bug.

-Bill