Discussion:
[CMake] makefile to CMakeLists.txt, possible?
Mr Shore
2010-08-19 13:02:49 UTC
Permalink
As we know most (old) open source projects are using makefile,

is there a tool to convert makefile to CMakeLists.txt ?
Eric Noulard
2010-08-19 13:16:55 UTC
Permalink
Post by Mr Shore
As we know most (old) open source projects are using makefile,
is there a tool to convert makefile to CMakeLists.txt ?
None I am aware of.

Some converters are listed on the Wiki:
http://www.cmake.org/Wiki/CMake#Converters_from_other_buildsystems_to_CMake

I have some very basic CMakeLists.txt from scratch generator written in python
but I should be near to useless for "real" project.

Is you target project "pure" makefile based or does it use autotools?
--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
Eric Noulard
2010-08-19 13:39:44 UTC
Permalink
Actually it's not my projects, but the well known projects like MySQL,
Firefox and so on..
CMakeifying this kind of project may not be an easy task, moreover my own
opinion is that you shoudl ensure that the project stakeholder may be
interested in
this effort.

In fact if they are not interested you may be wasting your time
following the evolution
of the officially supported build system.

I do not think you'll find a useful converter tool for this kind of
project and even if
some tool may do say 80% of the work you'll end up diving in the
current build system
in order to manually do the remaining 20%.

Why would you want to CMakeify Firefox or MySQL?
--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
David Cole
2010-08-19 13:50:16 UTC
Permalink
MySQL already builds with CMake, currently.

For example, search for cmake on this page:
http://dev.mysql.com/doc/refman/5.1/en/connector-c-building.html
Post by Eric Noulard
Actually it's not my projects, but the well known projects like MySQL,
Firefox and so on..
CMakeifying this kind of project may not be an easy task, moreover my own
opinion is that you shoudl ensure that the project stakeholder may be
interested in
this effort.
In fact if they are not interested you may be wasting your time
following the evolution
of the officially supported build system.
I do not think you'll find a useful converter tool for this kind of
project and even if
some tool may do say 80% of the work you'll end up diving in the
current build system
in order to manually do the remaining 20%.
Why would you want to CMakeify Firefox or MySQL?
--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.cmake.org/Wiki/CMake_FAQ
http://www.cmake.org/mailman/listinfo/cmake
Eric Noulard
2010-08-19 14:12:00 UTC
Permalink
Basically, cmake parsor can understand makefile, that's why it can generate
makefile from CMakeLists.txt,
As far as I know CMake does NOT understand Makefile it does **generates**
makefiles which is totally different story.

Again AFAIK CMake is never **reading** a Makefile.

I invite you to verify my feeling from the CMake source code.
Just the other way around,why can't it 100% convert a makefile to
CMakeLists.txt?
Just as with foreign language, because of the semantic.

Usually I am able to say what I want in english without trouble.
But give me some litterature in english to translate back to my
native language and I'll get stuck after 2 pages or 2 lines depending
on the author.
--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
"Roman Wüger" @mac.com
2010-08-19 17:11:54 UTC
Permalink
CMake is a Makefile-Generator an nothing else

Why would you write the Makefiles by yourself?

Best Regards
NoRulez
Post by Eric Noulard
Basically, cmake parsor can understand makefile, that's why it can generate
makefile from CMakeLists.txt,
As far as I know CMake does NOT understand Makefile it does **generates**
makefiles which is totally different story.
Again AFAIK CMake is never **reading** a Makefile.
I invite you to verify my feeling from the CMake source code.
Just the other way around,why can't it 100% convert a makefile to
CMakeLists.txt?
Just as with foreign language, because of the semantic.
Usually I am able to say what I want in english without trouble.
But give me some litterature in english to translate back to my
native language and I'll get stuck after 2 pages or 2 lines depending
on the author.
--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
http://www.cmake.org/mailman/listinfo/cmake
Chiheng Xu
2010-09-02 05:17:35 UTC
Permalink
On Fri, Aug 20, 2010 at 1:11 AM, <"\"Roman Wüger\"
Post by "Roman Wüger" @mac.com
CMake is a Makefile-Generator an nothing else
Why would you write the Makefiles by yourself?
Perhaps some people don't like the time consuming invocation of CMake.

Maybe CMake can generate relocatable Makefiles only containing
relative paths for several major platforms, and developers "cache"
the generated relocatable Makefiles in version control systems. Then
builders don't need to invoke CMake.
--
Chiheng Xu
Wuhan,China
J Decker
2010-09-02 05:58:51 UTC
Permalink
On Fri, Aug 20, 2010 at 1:11 AM,  <"\"Roman Wüger\"
Post by "Roman Wüger" @mac.com
CMake is a Makefile-Generator an nothing else
Why would you write the Makefiles by yourself?
Perhaps some people don't like the time consuming invocation of CMake.
Maybe CMake can generate relocatable Makefiles only containing
relative paths for several major platforms,  and developers "cache"
the generated relocatable Makefiles in version control systems.  Then
builders don't need to invoke CMake.
Then why bother writing CMake files? just use make.
--
Chiheng Xu
Wuhan,China
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
http://www.cmake.org/mailman/listinfo/cmake
Chris Hillery
2010-09-02 06:02:40 UTC
Permalink
Post by Chiheng Xu
On Fri, Aug 20, 2010 at 1:11 AM, <"\"Roman Wüger\"
Post by "Roman Wüger" @mac.com
CMake is a Makefile-Generator an nothing else
Why would you write the Makefiles by yourself?
Perhaps some people don't like the time consuming invocation of CMake.
Maybe CMake can generate relocatable Makefiles only containing
relative paths for several major platforms, and developers "cache"
the generated relocatable Makefiles in version control systems. Then
builders don't need to invoke CMake.
That's technically possible, but I feel sure it would introduce far more
problems than it would solve.

It would also rather spectacularly miss the point of CMake. You could
equally well say "Maybe gcc could generate assembly code for several major
platforms, and then you could store the generated assembly files in version
control..."

Ceej
aka Chris Hillery
Chiheng Xu
2010-09-02 07:15:38 UTC
Permalink
Post by Chris Hillery
On Fri, Aug 20, 2010 at 1:11 AM,  <"\"Roman Wüger\"
Post by "Roman Wüger" @mac.com
CMake is a Makefile-Generator an nothing else
Why would you write the Makefiles by yourself?
Perhaps some people don't like the time consuming invocation of CMake.
Maybe CMake can generate relocatable Makefiles only containing
relative paths for several major platforms,  and developers "cache"
the generated relocatable Makefiles in version control systems.  Then
builders don't need to invoke CMake.
That's technically possible, but I feel sure it would introduce far more
problems than it would solve.
It would also rather spectacularly miss the point of CMake. You could
equally well say "Maybe gcc could generate assembly code for several major
platforms, and then you could store the generated assembly files in version
control..."
CMake is a very great tool. But its drawback is also obvious. It
will consume large amount of time to generate Makefiles every time you
want to build, especially for ultra large projects. Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized. So, the "caching" of Makefiles seems benefitial for
current and future mutil-cores systems.

Normal developers seldomly touch CMakeLists.txt. So, in very rare
condition, you need to regenerate the Makefiles.
--
Chiheng Xu
Wuhan,China
Mike McQuaid
2010-09-02 07:32:38 UTC
Permalink
Post by Chiheng Xu
CMake is a very great tool. But its drawback is also obvious. It
will consume large amount of time to generate Makefiles every time you
want to build, especially for ultra large projects. Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized. So, the "caching" of Makefiles seems benefitial for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt. So, in very rare
condition, you need to regenerate the Makefiles.
That's why the files are cached on your local machine. I've seen some pretty huge CMake projects and I've never seen it take more than a few minutes to generate the Makefiles. I don't see why you have to keep them in version control.

Most projects I've worked on don't keep any generated files in version control, be it from the makefile generator, code generator or binary files.

--
Cheers,
Mike McQuaid
http://mikemcquaid.com
Chiheng Xu
2010-09-02 07:49:06 UTC
Permalink
Post by Mike McQuaid
CMake is a very great tool.    But its drawback is also obvious.    It
will consume large amount of time to generate Makefiles every time you
want to build,  especially for ultra large projects.   Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized.   So, the "caching" of Makefiles seems benefitial  for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt.  So,  in very rare
condition, you need to regenerate the Makefiles.
That's why the files are cached on your local machine. I've seen some pretty huge CMake projects and I've never seen it take more than a few minutes to generate the Makefiles. I don't see why you have to keep them in version control.
Most projects I've worked on don't keep any generated files in version control, be it from the makefile generator, code generator or binary files.
Suppose you have a ultra large project, it will consume 5 minutes to
CMake, 2 hours to build serially. If you have an 64 Cores ccNUMA
systems, like Xeon 7500 (8 Cores * 8), theretically, you will have a
60+ accelaration in parallel build. Theretically, it will consume
less than 2 minutes to parallelly build, but it will also consume 5
minutes to serially CMake. So, If you "cache" the Makefiles, it will
only consume less than 2 minutes to build.
--
Chiheng Xu
Wuhan,China
Michael Wild
2010-09-02 08:04:09 UTC
Permalink
Post by Chiheng Xu
Post by Mike McQuaid
Post by Chiheng Xu
CMake is a very great tool. But its drawback is also obvious. It
will consume large amount of time to generate Makefiles every time you
want to build, especially for ultra large projects. Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized. So, the "caching" of Makefiles seems benefitial for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt. So, in very rare
condition, you need to regenerate the Makefiles.
That's why the files are cached on your local machine. I've seen some pretty huge CMake projects and I've never seen it take more than a few minutes to generate the Makefiles. I don't see why you have to keep them in version control.
Most projects I've worked on don't keep any generated files in version control, be it from the makefile generator, code generator or binary files.
Suppose you have a ultra large project, it will consume 5 minutes to
CMake, 2 hours to build serially. If you have an 64 Cores ccNUMA
systems, like Xeon 7500 (8 Cores * 8), theretically, you will have a
60+ accelaration in parallel build. Theretically, it will consume
less than 2 minutes to parallelly build, but it will also consume 5
minutes to serially CMake. So, If you "cache" the Makefiles, it will
only consume less than 2 minutes to build.
2 hours serial build? You consider that ultra large? That is medium sized at best. And if it takes 5 minutes to run CMake there must be some pretty inefficient stuff in that CMake code of yours... Also, I really doubt you will get 60+ speedup (due to bottlenecks in your CPU, memory and disk).

And the real problem are not "fresh" builds. They usually take much longer than it takes CMake to run. The problems arise if you make a small modification, requiring CMake to re-run, but only a few files are being recompiled. Those are the builds that slow a developer down, because that's what he's doing most of the time. You don't build from scratch every time (if you do, don't worry about the time it takes CMake to run...).

Also, consider that maintaining hand-crafted Makefile's would require much more of your time ;-)

My 2c.

Michael
Mike McQuaid
2010-09-02 08:04:55 UTC
Permalink
Post by Chiheng Xu
Suppose you have a ultra large project, it will consume 5 minutes to
CMake, 2 hours to build serially. If you have an 64 Cores ccNUMA
systems, like Xeon 7500 (8 Cores * 8), theretically, you will have a
60+ accelaration in parallel build. Theretically, it will consume
less than 2 minutes to parallelly build, but it will also consume 5
minutes to serially CMake. So, If you "cache" the Makefiles, it will
only consume less than 2 minutes to build.
First of all, I'm not aware of any development teams using that level of hardware and I doubt yours are either. Secondly, you'll find that as the CPU power grows, building is limited by disk IO and memory bandwidth rather than CPU power.

Regardless, what you are saying is that it will take 2 minutes to build EACH TIME and 5 minutes to run CMake ONCE (as you said developers rarely ever touch CMake files).

If you want to use Makefiles, fine, but without benchmarks your argument is pretty meaningless FUD against CMake and other makefile generators.

--
Cheers,
Mike McQuaid
http://mikemcquaid.com
Michael Wild
2010-09-02 08:08:04 UTC
Permalink
Post by Mike McQuaid
Post by Chiheng Xu
Suppose you have a ultra large project, it will consume 5 minutes to
CMake, 2 hours to build serially. If you have an 64 Cores ccNUMA
systems, like Xeon 7500 (8 Cores * 8), theretically, you will have a
60+ accelaration in parallel build. Theretically, it will consume
less than 2 minutes to parallelly build, but it will also consume 5
minutes to serially CMake. So, If you "cache" the Makefiles, it will
only consume less than 2 minutes to build.
First of all, I'm not aware of any development teams using that level of hardware and I doubt yours are either. Secondly, you'll find that as the CPU power grows, building is limited by disk IO and memory bandwidth rather than CPU power.
Regardless, what you are saying is that it will take 2 minutes to build EACH TIME and 5 minutes to run CMake ONCE (as you said developers rarely ever touch CMake files).
If you want to use Makefiles, fine, but without benchmarks your argument is pretty meaningless FUD against CMake and other makefile generators.
Oh, and yes: Ever timed "gcc -M" which is the usual approach to dependency-generation of hand-crafted Makefiles? THAT is slow. ;-)

Michael
Jed Brown
2010-09-02 08:56:02 UTC
Permalink
Post by Michael Wild
Ever timed "gcc -M" which is the usual approach to dependency-generation of hand-crafted Makefiles? THAT is slow. ;-)
It's also correct, which requires preprocessing all the headers.

gcc -MMD will do the same as a side-effect of the build so it's
basically free. Tup generates this graph by tracing the system calls
made by the compiler, but it requires dynamic loading. The problem
with these solutions is that they are not sufficiently portable, until
every compiler vendor supports -MMD, or ever potential build system
has dynamic loading, we have to live with a slow and inaccurate, or
very slow, method.

Jed
Raymond Wan
2010-09-02 14:11:05 UTC
Permalink
Hi Chiheng,
CMake is a very great tool.    But its drawback is also obvious.    It
will consume large amount of time to generate Makefiles every time you
want to build,  especially for ultra large projects.   Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized.   So, the "caching" of Makefiles seems benefitial  for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt.  So,  in very rare
condition, you need to regenerate the Makefiles.
As someone who just started using CMake (quite literally a few weeks
ago), I'm somewhat interested in your arguments. Personally, I don't
see many drawbacks and I must be missing something if they are
"obvious".

It is true that while developing software, using CMake requires an
extra step. For example, when you add a C++ source file, you'll need
to run cmake and then make. But after development, having your
software being usable across multiple platforms seems to be worth it.
If later on, you want to add a file, you don't have to modify the
Linux Makefile, the Mac Makefile, and the Windows Makefile. IMHO,
CMake saves the time of the developers.

After many years of writing Makefile files by hand, Autotools seemed
like moving 3 steps forward and 2 steps back. CMake seems to be a big
improvement. But CMake makes Makefiles and if you don't like it, you
can always stick with just Makefiles. When you get tired of that,
then you'll be glad that both Autotools and CMake are available
options for you...

Ray

PS: Your example of parallel compilation doesn't take into account
that only the files that change need to be recompiled. And then when
the executable is to be re-linked, the object files have to be placed
on the same disk, etc. I don't know if CMake supports parallel
compilation (i.e., for n processors, generate n Makefiles so that each
CPU is responsible for just their parts), but that would be cool...
:-)
Alexander Neundorf
2010-09-02 16:47:13 UTC
Permalink
Post by Raymond Wan
Hi Chiheng,
CMake is a very great tool.    But its drawback is also obvious.    It
will consume large amount of time to generate Makefiles every time you
want to build,  especially for ultra large projects.   Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized.   So, the "caching" of Makefiles seems benefitial  for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt.  So,  in very rare
condition, you need to regenerate the Makefiles.
As someone who just started using CMake (quite literally a few weeks
ago), I'm somewhat interested in your arguments. Personally, I don't
see many drawbacks and I must be missing something if they are
"obvious".
Generated makefiles can't be portable e.g. because they contain include paths
and paths to libraries which will be linked which can differ on each system.
Also, the generated makefiles contain calls to cmake e.g. for the progress
report.
Also, it is quite common to use cmake -E for cross platform commands e.g. in
add_custom_command() or add_custom_target(), which results in rules in the
makefiles which call cmake to do something, e.-g. copy a file or something.
Post by Raymond Wan
It is true that while developing software, using CMake requires an
extra step. For example, when you add a C++ source file, you'll need
to run cmake and then make.
If cmake has already generated makefiles for you, you don't need to invoke it
manually again after editing CMakeLists.txt, the generated makefiles include
this dependency and rerun cmake automatically for you.


Nevertheless, some way to make something in cmake parallel would be nice :-)
But at least for me it's not obvious how to do that.

Alex
Chiheng Xu
2010-09-03 08:35:58 UTC
Permalink
On Fri, Sep 3, 2010 at 12:47 AM, Alexander Neundorf
Post by Alexander Neundorf
Generated makefiles can't be portable e.g. because they contain include paths
and paths to libraries which will be linked which can differ on each system.
Also, the generated makefiles contain calls to cmake e.g. for the progress
report.
Also, it is quite common to use cmake -E for cross platform commands e.g. in
add_custom_command() or add_custom_target(), which results in rules in the
makefiles which call cmake to do something,  e.-g. copy a file or something.
Post by Raymond Wan
It is true that while developing software, using CMake requires an
extra step.  For example, when you add a C++ source file, you'll need
to run cmake and then make.
If cmake has already generated makefiles for you, you don't need to invoke it
manually again after editing CMakeLists.txt, the generated makefiles include
this dependency and rerun cmake automatically for you.
Nevertheless, some way to make something in cmake parallel would be nice :-)
But at least for me it's not obvious how to do that.
Why CMake can't run as make ?

CMake parse CMakeLists.txt, generate Makefile. Then, make parse the
Makefile, and execute(serially or parallelly) shells, shells run
comands(compiler, etc) to build.

Perhaps, CMake can parse CMakeLists.txt, and directly run
comands(compiler, etc) to build, without gererating Makefile,
invoking make, invoking shells.
--
Chiheng Xu
Wuhan,China
Chris Hillery
2010-09-03 09:02:00 UTC
Permalink
Post by Chiheng Xu
Perhaps, CMake can parse CMakeLists.txt, and directly run
comands(compiler, etc) to build, without gererating Makefile,
invoking make, invoking shells.
Three answers, just off the top of my head:

1. Why re-invent the wheel? Make already does a lot of things quite well.
CMake builds on that.

2. Remember that CMake doesn't only work with Make. It can generate Visual
Studio projects and drive several other build systems. CMake isn't a build
system itself; it's meant to provide a common mechanism to abstract code
organization away from the underlying build system.

3. There's a lot of things CMakeLists might do that aren't directly
build-related, such as creating test configurations and identifying
platform-specific dependencies. There's no need to go through all those
steps every time you just want to compile your latest change; it makes sense
to have a separate configuration step for those.

There have been several other projects which attempted to be a "better Make"
which is effectively what you're describing. There's nothing wrong with
that, but that isn't what CMake does. If that's what you want in a tool,
then I think you'll be happier finding a tool with that goal - always pick
the right tool for the job at hand!

Ceej
aka Chris Hillery
Alexander Neundorf
2010-09-03 17:45:11 UTC
Permalink
Post by Alexander Neundorf
Post by Raymond Wan
Hi Chiheng,
CMake is a very great tool.    But its drawback is also obvious.    It
will consume large amount of time to generate Makefiles every time you
want to build,  especially for ultra large projects.   Building can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized.   So, the "caching" of Makefiles seems benefitial  for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt.  So,  in very rare
condition, you need to regenerate the Makefiles.
As someone who just started using CMake (quite literally a few weeks
ago), I'm somewhat interested in your arguments. Personally, I don't
see many drawbacks and I must be missing something if they are
"obvious".
Generated makefiles can't be portable e.g. because they contain include
paths and paths to libraries which will be linked which can differ on each
system. Also, the generated makefiles contain calls to cmake e.g. for the
progress report.
Also, it is quite common to use cmake -E for cross platform commands e.g.
in add_custom_command() or add_custom_target(), which results in rules in
the makefiles which call cmake to do something, e.-g. copy a file or
something.
Post by Raymond Wan
It is true that while developing software, using CMake requires an
extra step. For example, when you add a C++ source file, you'll need
to run cmake and then make.
If cmake has already generated makefiles for you, you don't need to invoke
it manually again after editing CMakeLists.txt, the generated makefiles
include this dependency and rerun cmake automatically for you.
Nevertheless, some way to make something in cmake parallel would be nice
:-) But at least for me it's not obvious how to do that.
First step would be to profile where cmake spends most of the time.

I would guess that there is a lot of I/O (where running multiple threads
probably would not help much), and probably string functions like strcmp(),
strlen() and the std::string ctor.
Internally cmake uses mostly std::strings, but a lot of functions in cmake
take char* strings as parameters, and often then convert them back to
std::strings. Adding variants which directly take std::string might make
things faster (IIRC std::strings have explicit lengths so a lot of strlen()
calls should be saved and comparing two strings should be also faster).

Beside that, maybe in the final generate-step writing of the output files
could be parallelized (like having one thread with a job queue to write one
file after the other while the main thread keeps processing), but this is
probably IO-bound anyway.

Alex
Michael Jackson
2010-09-03 18:34:00 UTC
Permalink
Post by Alexander Neundorf
Post by Alexander Neundorf
Post by Raymond Wan
Hi Chiheng,
Post by Chiheng Xu
CMake is a very great tool. But its drawback is also
obvious. It
will consume large amount of time to generate Makefiles every time you
want to build, especially for ultra large projects. Building
can be
parallelized, but the CMaking, like runing configure scripts, can't be
parallelized. So, the "caching" of Makefiles seems benefitial
for
current and future mutil-cores systems.
Normal developers seldomly touch CMakeLists.txt. So, in very rare
condition, you need to regenerate the Makefiles.
As someone who just started using CMake (quite literally a few weeks
ago), I'm somewhat interested in your arguments. Personally, I don't
see many drawbacks and I must be missing something if they are
"obvious".
Generated makefiles can't be portable e.g. because they contain include
paths and paths to libraries which will be linked which can differ on each
system. Also, the generated makefiles contain calls to cmake e.g. for the
progress report.
Also, it is quite common to use cmake -E for cross platform
commands e.g.
in add_custom_command() or add_custom_target(), which results in rules in
the makefiles which call cmake to do something, e.-g. copy a file or
something.
Post by Raymond Wan
It is true that while developing software, using CMake requires an
extra step. For example, when you add a C++ source file, you'll need
to run cmake and then make.
If cmake has already generated makefiles for you, you don't need to invoke
it manually again after editing CMakeLists.txt, the generated
makefiles
include this dependency and rerun cmake automatically for you.
Nevertheless, some way to make something in cmake parallel would be nice
:-) But at least for me it's not obvious how to do that.
First step would be to profile where cmake spends most of the time.
I would guess that there is a lot of I/O (where running multiple threads
probably would not help much), and probably string functions like strcmp(),
strlen() and the std::string ctor.
Internally cmake uses mostly std::strings, but a lot of functions in cmake
take char* strings as parameters, and often then convert them back to
std::strings. Adding variants which directly take std::string might make
things faster (IIRC std::strings have explicit lengths so a lot of strlen()
calls should be saved and comparing two strings should be also
faster).
Beside that, maybe in the final generate-step writing of the output files
could be parallelized (like having one thread with a job queue to write one
file after the other while the main thread keeps processing), but this is
probably IO-bound anyway.
Alex
I was just thinking about this the other day. One specialized area
that would be helped by threads would be in some project configuration
where we have CMake looking for about 20 different headers and another
20 or so types. Generally none of this is dependent on the other.
Something where I could have a
"parallel_section()"/"end_parallel_section()" section where I could
list all the tests and CMake could run them in parallel if the builder
allows it or there are enough cpu resources available.
Just a thought.
___________________________________________________________
Mike Jackson www.bluequartz.net
Principal Software Engineer ***@bluequartz.net
BlueQuartz Software Dayton, Ohio
Alexander Neundorf
2010-09-03 19:19:28 UTC
Permalink
On Friday 03 September 2010, Michael Jackson wrote:
...
Post by Michael Jackson
I was just thinking about this the other day. One specialized area
that would be helped by threads would be in some project configuration
where we have CMake looking for about 20 different headers and another
20 or so types. Generally none of this is dependent on the other.
Something where I could have a
"parallel_section()"/"end_parallel_section()" section where I could
list all the tests and CMake could run them in parallel if the builder
allows it or there are enough cpu resources available.
Just a thought.
Maybe really some parallel capability could be added specifically to the
try_compile() command.
Which calls cmMakefile::TryCompile(), which calls
cmGlobalGenerator::TryCompile(), which ends up in cmGlobalGenerator::Build(),
which finally calls

if (!cmSystemTools::RunSingleCommand(command, outputPtr,
&retVal, 0, verbose, timeout))

Maybe just this one call could be put into threads somehow ?

Alex
Christian Ehrlicher
2010-09-03 19:33:09 UTC
Permalink
Post by Alexander Neundorf
...
Post by Michael Jackson
I was just thinking about this the other day. One specialized area
that would be helped by threads would be in some project configuration
where we have CMake looking for about 20 different headers and another
20 or so types. Generally none of this is dependent on the other.
Something where I could have a
"parallel_section()"/"end_parallel_section()" section where I could
list all the tests and CMake could run them in parallel if the builder
allows it or there are enough cpu resources available.
Just a thought.
Maybe really some parallel capability could be added specifically to the
try_compile() command.
Which calls cmMakefile::TryCompile(), which calls
cmGlobalGenerator::TryCompile(), which ends up in
cmGlobalGenerator::Build(), which finally calls
if (!cmSystemTools::RunSingleCommand(command, outputPtr,
&retVal, 0, verbose, timeout))
Maybe just this one call could be put into threads somehow ?
A short non-representation 'valgrind -tool=callgrind cmake .' shows that when
no configure checks need to be done, 25 percent of the time is used for
strstr() / SystemTools::ReplaceString() ... wouldn't this be also a good
point to start optimizing?


Christian
Alexander Neundorf
2010-09-03 20:33:02 UTC
Permalink
On Friday 03 September 2010, Christian Ehrlicher wrote:
...
Post by Christian Ehrlicher
A short non-representation 'valgrind -tool=callgrind cmake .' shows that
when no configure checks need to be done, 25 percent of the time is used
for strstr() / SystemTools::ReplaceString() ... wouldn't this be also a
good point to start optimizing?
Of course :-)

Alex
Eric Noulard
2010-09-02 06:21:20 UTC
Permalink
On Fri, Aug 20, 2010 at 1:11 AM,  <"\"Roman Wüger\"
Post by "Roman Wüger" @mac.com
CMake is a Makefile-Generator an nothing else
Why would you write the Makefiles by yourself?
Perhaps some people don't like the time consuming invocation of CMake.
Maybe CMake can generate relocatable Makefiles only containing
relative paths for several major platforms,  and developers "cache"
the generated relocatable Makefiles in version control systems.  Then
builders don't need to invoke CMake.
If you dig the ML you'll find several [very] good reasons why
it is not as easy as it seems for CMake to generate a set of
"relocatable" build files which does not need CMake anymore.
(currenlty generated Makefile (or other project files) may invoke CMake)

Trying to do that is not only difficult but will limit current CMake usability,
i.e. a lot of thing may happen "at cmake time" but you may want to
"invoke directly or indirectly cmake at build time too".

I'm not saying it wouldn't be theoretically possible nor practically useful
but that wasn't built in CMake design and now it seems difficult to do
with "not so much foreseen benefits".

As a side note, in the past, I did have to maintain a set of "portable
makefile" system
for 5 to 10 different platforms. After a while it was decided that we
should enforce
GNU make on all of them otherwise maintenance would be too time consuming.

After that I did use autotools for 3-4 years, then after 6+years of CMake usage
my point of view is that it's far easier to teach some of your
developper/builder how
to "basically" use CMake, than trying to maintain a portable build
system for all of them
using makefiles and/or autotools. Autotools may be OK if you do not
support windows platform.

I save a **lot of time** going from bare makefiles, to autotools and then CMake.

For those "bare [but respectable] builders" that do not want to touch CMakery
I usually try to craft some automated script like the one proposed here:
http://www.cmake.org/pipermail/cmake/2010-August/039270.html
--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
Diablo 666
2010-09-02 14:44:51 UTC
Permalink
Hi,
Post by Raymond Wan
For example, when you add a C++ source file,
you'll need to run cmake and then make.
make will automatically re-run cmake, if the cmake files have changed.
I guess, there will be some IDEs automatically generating cmake files in the future.
Post by Raymond Wan
generate n Makefiles so that each CPU is responsible for just their parts
you might want to try make -j. I guess, this is exactly what you are searching for.

Best regards,
Andreas
Bill Hoffman
2010-09-02 14:54:49 UTC
Permalink
Post by Diablo 666
Hi,
Post by Raymond Wan
For example, when you add a C++ source file,
you'll need to run cmake and then make.
make will automatically re-run cmake, if the cmake files have changed.
I guess, there will be some IDEs automatically generating cmake files in the future.
The IDE's re-run CMake as well.


-Bill
Michael Wild
2010-09-02 14:57:29 UTC
Permalink
Post by Bill Hoffman
Post by Diablo 666
Hi,
Post by Raymond Wan
For example, when you add a C++ source file,
you'll need to run cmake and then make.
make will automatically re-run cmake, if the cmake files have changed.
I guess, there will be some IDEs automatically generating cmake files in the future.
The IDE's re-run CMake as well.
-Bill
I think what Andreas meant is that he expects IDE's to use CMake as their native build system and auto-generate the CMake code.

Michael
Raymond Wan
2010-09-06 05:22:41 UTC
Permalink
Hi Andreas,

Thanks for the reply and the two points that you raised.

Yes, "make -j" is what I was thinking of. It just didn't occur to me
that it would be make's job; thought separating compilation into
multiple processes has be done at the point the Makefile is
created...guess I was wrong...

Thanks and sorry for the delay; was away from e-mail for a few days
and I've noticed the thread has gone away already... Just wanted to
say "thanks"!

Ray
Post by Diablo 666
Hi,
Post by Raymond Wan
For example, when you add a C++ source file,
you'll need to run cmake and then make.
make will automatically re-run cmake, if the cmake files have changed.
I guess, there will be some IDEs automatically generating cmake files in the future.
Post by Raymond Wan
generate n Makefiles so that each CPU is responsible for just their parts
you might want to try make -j. I guess, this is exactly what you are searching for.
Best regards,
Andreas
Diablo 666
2010-09-02 15:02:02 UTC
Permalink
Post by Michael Wild
I think what Andreas meant is that he expects IDE's to use CMake
as their native build system and auto-generate the CMake code.
Exactly. AFAIK KDevelop 4 is actually building cmake files. But I didn't
see a working version of it, yet :(
Michael Wild
2010-09-02 15:04:25 UTC
Permalink
Post by Diablo 666
Post by Michael Wild
I think what Andreas meant is that he expects IDE's to use CMake
as their native build system and auto-generate the CMake code.
Exactly. AFAIK KDevelop 4 is actually building cmake files. But I didn't
see a working version of it, yet :(
I thought you had to manage the CMakeLists.txt yourself...

Michael
Andreas Pakulat
2010-09-02 15:31:02 UTC
Permalink
Post by Diablo 666
Post by Michael Wild
I think what Andreas meant is that he expects IDE's to use CMake
as their native build system and auto-generate the CMake code.
Exactly. AFAIK KDevelop 4 is actually building cmake files.
It doesn't, it tries to help you write them and even suggests changes when
you create classes or rename files. But it doesn't generate them from
ground up or completely manages them in some GUI.
Post by Diablo 666
But I didn't see a working version of it, yet :(
http://download.kde.org/download.php?url=stable/kdevelop/4.0.2/src/

In a day or to (until then replace 4.0.2 with 4.0.1).

Andreas
--
You will be awarded some great honor.
Loading...