[CMake] Custom-Targets with unknown outputs

[Christoph Cullmann]

Hi,

what's a good way to integrate a custom target that works like the following:

We have a generator, that takes some known files (like bison or flex) but
instead of generating few known ones, it will generate a full subdir of
source files with not predictable names. Atm we generate makefiles for this
subdir in the generator and call make from the outside for this subdir + link
in the generated static lib. Bad thing is, this isn't that platform
independend.

Is there a sane way to handle this with cmake? I thought of creating a
CMakeLists.txt in the subdir, but how to include/subdir it, as it will not
exist at the first run of cmake but be created while building.

cu
Christoph

[Alan W. Irwin]

Post by Christoph Cullmann
Hi,
We have a generator, that takes some known files (like bison or flex) but
instead of generating few known ones, it will generate a full subdir of
source files with not predictable names.

That's an intriguing question. Assuming the above is already done with
ADD_CUSTOM_COMMAND, how about adding another COMMAND (see the documentation
of ADD_CUSTOM_COMMAND for the syntax for multiple COMMANDs) that runs CMake
-E touch filename at run time? This will create (or change create time) for
an empty output file with a fixed unique name of your choice that can be
used as a file dependency for a custom target.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Alan W. Irwin]

Hi Christoph:

I am going to put this back on the list because others may have better insight
than I do.

Still, how can I at all include the generated sources into my build? I don't
want to use the makefiles generated but better convert them to cmake, too.

I have now reviewed your original question, and I think I understand it
better. At build time you want to generate a large number of source files
with unpredictable names and then use some CMake rule to build a library
using that source. If you had predicatable names the problem is
straightforward, but with unpredictable ones, I believe doing all that at
build time with CMake is tough/impossible, but I would be happy to be proved
wrong by someone else here.

One alternative I was thinking of was to generate your source at cmake time
with bison/flex using EXECUTE_PROCESS. The you can collect all the
generated file names into a list variable using file(GLOB ...) or
file(GLOB_RECURSE...) and then use ADD_LIBRARY with that list variable to
create a run-time rule to generate the library. But I don't recommend that
alternative since the cmake step would run bison/flex every time to generate
(or regenerate) your source, and the dependencies on the files used by
bison/flex would be completely ignored.

Are you sure the generated source file names are completely unpredictable?
Unless some random generator is being used, those names are probably
predictable, and it might be worth your while to look further into figuring
out how you can do that.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Clinton Stimpson]

Would it work to use ctest in a custom comand to build the generated
source files?
I'm thinking something similar to how the VTK Examples directory is
built as a separate project.

Clint

Post by Alan W. Irwin
I am going to put this back on the list because others may have better insight
than I do.

Still, how can I at all include the generated sources into my build? I don't
want to use the makefiles generated but better convert them to cmake, too.

I have now reviewed your original question, and I think I understand it
better. At build time you want to generate a large number of source files
with unpredictable names and then use some CMake rule to build a library
using that source. If you had predicatable names the problem is
straightforward, but with unpredictable ones, I believe doing all that at
build time with CMake is tough/impossible, but I would be happy to be proved
wrong by someone else here.
One alternative I was thinking of was to generate your source at cmake time
with bison/flex using EXECUTE_PROCESS. The you can collect all the
generated file names into a list variable using file(GLOB ...) or
file(GLOB_RECURSE...) and then use ADD_LIBRARY with that list variable to
create a run-time rule to generate the library. But I don't recommend that
alternative since the cmake step would run bison/flex every time to generate
(or regenerate) your source, and the dependencies on the files used by
bison/flex would be completely ignored.
Are you sure the generated source file names are completely
unpredictable?
Unless some random generator is being used, those names are probably
predictable, and it might be worth your while to look further into figuring
out how you can do that.
Alan
__________________________
Alan W. Irwin
Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).
Programming affiliations with the FreeEOS equation-of-state
implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________
Linux-powered Science
__________________________
_______________________________________________
CMake mailing list
http://www.cmake.org/mailman/listinfo/cmake

[Christoph Cullmann]

Post by Clinton Stimpson
Would it work to use ctest in a custom comand to build the generated
source files?
I'm thinking something similar to how the VTK Examples directory is
built as a separate project.

Hmm, is it then possible that other targets depend on the result objects or
libs build by ctest? And how are flags passed over and all that?

cu
Christoph

[Clinton Stimpson]

Post by Christoph Cullmann

Post by Clinton Stimpson
Would it work to use ctest in a custom comand to build the generated
source files?
I'm thinking something similar to how the VTK Examples directory is
built as a separate project.

Hmm, is it then possible that other targets depend on the result objects or
libs build by ctest? And how are flags passed over and all that?

See attached CMakeLists.txt file (modified from VTK examples) that shows
how to do that (dependencies, passing flags, etc..)
It would give you the chance to do a glob (or whatever you had in mind)
at build time to gather up the source files to compile into a library.
It doesn't include your custom command for generating the sources.

Clint

[Christoph Cullmann]

Post by Alan W. Irwin
Are you sure the generated source file names are completely unpredictable?
Unless some random generator is being used, those names are probably
predictable, and it might be worth your while to look further into figuring
out how you can do that.

The only thing predictable is the output directory (where atm a Makefile is
placed which we use with gmake + link the resulting library in the toplevel
makefile, as that name is known, too.). We can't predict them, as the code
generator generates one file per datatype and the datatypes are dynamic
dependend of the input file. To perdict them, in fact, the generator would
have to run.

[Martin Apel]

Post by Christoph Cullmann

Post by Alan W. Irwin
Are you sure the generated source file names are completely unpredictable?
Unless some random generator is being used, those names are probably
predictable, and it might be worth your while to look further into figuring
out how you can do that.

The only thing predictable is the output directory (where atm a Makefile is
placed which we use with gmake + link the resulting library in the toplevel
makefile, as that name is known, too.). We can't predict them, as the code
generator generates one file per datatype and the datatypes are dynamic
dependend of the input file. To perdict them, in fact, the generator would
have to run.

That's what I did for a similar problem: I modified the generator to
output a list of generated files, when called with a special command
line switch.
In this case the generator only generates the file list, not the files
themselves. I ran this via EXECUTE_PROCESS at configuration time, so you
have
the names of the generated files in a variable.
Of course this only works if you are able to modify the generator.

Martin
--

[Christoph Cullmann]

Post by Martin Apel
That's what I did for a similar problem: I modified the generator to
output a list of generated files, when called with a special command
line switch.
In this case the generator only generates the file list, not the files
themselves. I ran this via EXECUTE_PROCESS at configuration time, so you
have
the names of the generated files in a variable.
Of course this only works if you are able to modify the generator.

But then you must reconfigure on each change of the input files for the
generator, which is not that nice imho and atm with handcrafted makefiles not
needed, as the generator call will depend on it's input files.