rake help

[Mark Probert]

Hi ..

I have just started using the excellent Rake tool (thanks, Jim!) and I am at a
bit of a loss on how to proceed. I am attempting to create unit test for
some C++ code I am creating, using the cxxtest tool.

cxxtest has its tests contained in a .h file. These are then parsed by the
tool to give your .cpp file. This is then complied and linked with the
object file.

So, assuming I have my code in foo.cpp, my tests in foo_test.h, then the
sequence looks like:

$ c++ -c -o foo.o foo.cpp
$ cxxtest.pl -o foo_test.cpp foo_test.h
$ c++ -o foo_test foo_test.cpp foo.o

So, there are two issues that I am having problems with. The first is turning
the .h into a .cpp.

The second is how to get the test to conditionally depend on foo.o. I only
want to create foo.o if it isn't there. If it exists, then it will do for
the build. In reality, there will be multiple classes in each .o file, yet a
unit test per class.

Anyway, here is my rakefile, which isn't quite right. Giving:

$ rake
c++ -o unittest unittest scanner.o
c++: unittest: No such file or directory
rake aborted!

Many thanks,
-m.


# -!- Ruby -!-
#
# Rakefile for cocor_cpp
#

LINK_OBJ = "scanner.o"
task :default => [:unittest]

# --------
# Unit test harness for cocor
#
def ext(fn, newext)
fn.sub(/\.[^.]+$/, newext)
end

UT_SRC = [ "ts_buffer.h" ]
UT_CPP = UT_SRC.collect { |fn| ext(fn, ".cpp") }


UT_CPP.each do |utcpp|
utsrc = ext(utcpp, ".h")
file utcpp => [utsrc] do |t|
cxxopt = "--have-eh --error-printer"
sh( "cxxtestgen.pl #{cxxopt} -o #{utcpp} #{utsrc}" )
end
end


task :unittest => UT_CPP do |t|
exe = File.basename(t.name, '.*')
sh( "c++ -o #{exe} #{t.name} #{LINK_OBJ}" )
end

 

[Jim Weirich]

Mark Probert said:

I have just started using the excellent Rake tool (thanks, Jim!) and I am
at a
bit of a loss on how to proceed. I am attempting to create unit test for
some C++ code I am creating, using the cxxtest tool.

Hi Mark, Sorry it took me so long to respond to this. You probably have
solved this already, but I'll post for the general education of all Rake
Users everywhere (well, at least the ones on this list.

cxxtest has its tests contained in a .h file. These are then parsed by
the tool to give your .cpp file. This is then complied and linked
with the object file.

So, assuming I have my code in foo.cpp, my tests in foo_test.h, then the
sequence looks like:

$ c++ -c -o foo.o foo.cpp
$ cxxtest.pl -o foo_test.cpp foo_test.h
$ c++ -o foo_test foo_test.cpp foo.o

Ok, I'm going to work with this example rather than the Rakefile you
posted below. There should be enough info to apply it to your rakefile.

So, there are two issues that I am having problems with. The first is
turning the .h into a .cpp.

Not a problem ... You want to generate .cpp files from .h files with the
same base name. The easiest way is to express this as an explicit file
task:

file "foo_test.cpp" => ["foo_test.h"] do
sh %{cxxtest.pl -o foo_test.cpp foo_test.h}
end

The above says that file "foo_test.cpp" depends on a header file named
"foo_test.h", and that to create the .cpp file, all you need to do is run
the cxxtest.pl command with the given parameters.

Using the same logic, we can come up with tasks for the other two files as
well:

file "foo.o" => ["foo.cpp"] do
sh %{c++ -c -o foo.o foo.cpp}
end

file "foo_test" => ["foo_test.cpp", "foo.o"] do
sh %{c++ -o foo_test foo_test.cpp foo.o}
end

Those three file tasks together in a rake file will build foo_test
whenever you type "rake foo_test" at the command line. Supposedly you
want to run the tests as well. Simply add

task :unittest => ["foo_test"] do
sh "foo_test"
end

And if you want that to be the default task, then add:

task :default => [:unittest]

Take all of the above together and you get a working Rakefile. Now lets
fine tune it a bit.

First of all, the Rakefile is overly restrictive. Suppose you added
"bar_test.h" to the mix? You would have to add a task to build it and
modify existing tasks to depend upon it. Fortunately, you are using Ruby,
and you can create those tasks in a test...

FileList['*_test.h'].each do |source|
target = source.sub(/\.h$/, '.cpp')
file target => [fn] do
sh "cxxtest.pl -o #{target} #{source}"
end
end

We are assuming that the test header files contain end in "_test.h".
Modify according to your needs.

If you have 20 test headers, the above code will create 20 explicit tasks
for generating the test cpp files. This works great, however, there is
another way ... rules.

rule '.cpp' => '.h' do |t|
sh "cxxtest.pl -o #{t.name} #{t.source}"
end

This rule says that whenever you need a .cpp file, and you have a .h file
with the same name, here's how you would generate the .cpp file. It is
similar to generating the tasks explicitly (as we did in the loop above),
but instead of generating the tasks all at once it only does it on an as
needed basis.

In the same way, we can give a rule for compiling .cpp files into .o files.

rule '.o' => '.cpp' do |t|
sh "c++ -c -o #{t.name} #{t.source}"
end

I also like to create a bunch of file lists that describe the categories
of files. Here's what might work for you ...

CPP_SOURCE = FileList['*.cpp'].exclude('*_test.cpp')
TEST_HEADERS = FileList['*_test.h']

Now I want to combine these lists into a single list of all the object
files I will be dealing with ...

OBJ = FileList[
CPP_SOURCE.sub(/\.cpp$/, '.o'),
TEST_HEADERS.sub(/\.h$/, '.o'),
]

That leaves our final compile rule to look like this ...

file "foo_test" => OBJ do
sh "c++ -o foo_test #{OBJ}"
end

I'll show the complete Rakefile at the end and include some simple
clean/clobber rules.

The second is how to get the test to conditionally depend on foo.o. I
only want to create foo.o if it isn't there. [...]

Not a problem. Rake will only regenerate a file in a file task when the
file is out of date w.r.t. its dependencies.

Anyway, here is my rakefile, which isn't quite right. Giving:

$ rake
c++ -o unittest unittest scanner.o
c++: unittest: No such file or directory
rake aborted!

In your :unittest task, you are compiling and EXE, but give it t.name as
the file to compile. Since the task name is :unittest, there is no file
by that name.

Here's my Rakefile ...

----------------------------------------------------
# -*- ruby -*-

require 'rake/clean'

CLEAN.include('*.o')
CLOBBER.include('foo_test', 'foo_test.cpp')

CPP_SOURCE = FileList['*.cpp'].exclude(/test.cpp$/)
TEST_HEADERS = FileList['*_test.h']
OBJ = FileList[
CPP_SOURCE.sub(/\.cpp$/, '.o'),
TEST_HEADERS.sub(/\.h$/, '.o'),
]

task :default => :unittest

task :unittest => "foo_test" do
sh "foo_test"
end

file "foo_test" => OBJ do
sh "c++ -o foo_test #{OBJ}"
end

rule '.o' => '.cpp' do |t|
sh "c++ -c -o #{t.name} #{t.source}"
end

rule '.cpp' => '.h' do |t|
sh "cxxtest.pl -o #{t.name} #{t.source}"
end

 

[Mark Probert]

Hi ..

Many thanks, Jim. First, Rake is excellent! There is one part of the
scenario that I'd like to explore, and a comment.

rule '.cpp' => '.h' do |t|
sh "cxxtest.pl -o #{t.name} #{t.source}"
end

I did get around to reading more of the documentation and have ended up using
rules. My results where very similar to yours.

When I created th.h -> .cpp rule, it would run against all .h files that it
could find, rather than just the unit test ones. So I ended up with:

UT_EXE = [ "test1", "test2", "test3" ] # all have a .h file defining tests

rule '.cpp' => ['.h'] do |t|
name = t.name.sub(/\.[^.]+$/, "")
if UT_EXE.include?(name)
sh( "#{CXXTEST} #{CXXTESTOPT} -o #{t.name} #{t.source}" )
end
end

I have also found that it is useful to have a dependency list for the various
UTs, as not all of the UTs required all of the files of the application. So,
something like the following works:

UT_T1_DEP = [ "test1.cpp", "foo.o" ]
UT_T2_DEP = [ "test2.cpp", "bar.o" ]
UT_T3_DEP = [ "test3.cpp", "foo.o", "bar.o" ]

My UT drivers look like:

file "test1" => UT_T1_DEP do |t|
sh( "#{CC} #{CCOPT} -o #{t.name} #{t.prerequisites.join(' ')}" )
end

file "test2" => UT_T2_DEP do |t|
sh( "#{CC} #{CCOPT} -o #{t.name} #{t.prerequisites.join(' ')}" )
end

file "test3" => UT_T3_DEP do |t|
sh( "#{CC} #{CCOPT} -o #{t.name} #{t.prerequisites.join(' ')}" )
end

And the overall driver is a task:

task :testsuite => UT_EXE do |t|
puts "--> finished creating the tests"
end

The first question: is there a way to factor the UT drivers into a rule? I
can't seem to get the dependcies right.

Second: What is the best way of making this work in a sub-directory? Use
explicit paths in the UT_*_DEP lines to ensure the link is right? Have a
Rakefile in the UT director driven by one in the main directory?

Again, many thanks for an excellent tool! I have been playing with Make and
Jam for ages, and Rake has the potential to replace them all for me

Have you thought of doing the equivalent of Jambase and having a packaged
library of default rules that will be applied, based on the platform that
Rake is invoked on? So, if Linux, CC=gcc, if Win32 CC=cl, etc..

Regards,

 

[Sam Roberts]

Quoting (e-mail address removed), on Wed, Mar 30, 2005 at 07:11:05AM +0900:

I have just started using the excellent Rake tool (thanks, Jim!) and I am at a
bit of a loss on how to proceed. I am attempting to create unit test for
some C++ code I am creating, using the cxxtest tool.

cxxtest has its tests contained in a .h file. These are then parsed by the
tool to give your .cpp file. This is then complied and linked with the
object file.

So, assuming I have my code in foo.cpp, my tests in foo_test.h, then the
sequence looks like:

This is a little off-topic, but you might find that GNU make has had a
dozen years of optimization for doing this kind of stuff easily and
concisely:

--- makefile ---
default: test

TEST_H = $(wildcard *_test.h)
TEST_E = $(TEST_H:.h=)

exe: $(TEST_E)

test: exe
for t in $(TEST_E); do ./$$t; done

%.o: %.cpp
c++ -c -o $@ $<

%_test.cpp: %_test.h
cxxtest.pl -o $@ $<

%_test: %_test.o %.o
c++ -o $@ $^
----------------

This makefile enforces your naming convention, that the unit test for
BAR are in BAR_test.h, and the source is in BAR.cpp

I took the liberty of running the tests by default, as all good
makefiles should do.

Cheers,
Sam

 

[Jim Weirich]

I did get around to reading more of the documentation and have ended up
using rules. My results where very similar to yours.

When I created th.h -> .cpp rule, it would run against all .h files that it
could find, rather than just the unit test ones. So I ended up with:

Rules can be based on more than just file extensions. The target can be a
regular expression and the dependency can be an arbitrary lambda that does a
transform. The following (untested) might work for you ...

rule(/^test\d+\.cpp$/, '.h') do |t|
sh "#{CXXTEST} #{CXXTESTOPT} -o #{t.name} #{t.source}"
end

I have also found that it is useful to have a dependency list for the
various UTs, as not all of the UTs required all of the files of the
application. So, something like the following works:

Likewise, a pattern based rule might work here too (again, untested):

rule(/^test\d+$/, '.cpp') do |t|
sh "#{CC} #{CCOPT} -o #{t.name} #{t.prerequisites.join(' ')}"
end

And then the dependencies can be given explicitly.

file "test1" => ["test1.cpp", "foo.o"]
file "test2" => ["test2.cpp", "bar.o"]
file "test3" => ["test3.cpp", "foo.o", "bar.o"]

This become really powerful when you can write a little ruby to scan your
source for #includes and generate the dependencies automatically.

Hmmm ... now that I look at the above, I'm wondering if the explicit
dependencies will prevent the rules from triggering. Try it and let me know
if it doesn't work here.

The first question: is there a way to factor the UT drivers into a rule?
I can't seem to get the dependcies right.

Second: What is the best way of making this work in a sub-directory? Use
explicit paths in the UT_*_DEP lines to ensure the link is right? Have a
Rakefile in the UT director driven by one in the main directory?

I'm not a big fan of having one Rakefile invoke a Rakefile in a subdirectory.
I would rather see one Rakefile control the whole process. (You can get
finer grain dependencies that way). An alternative would be to place
Rakefile fragments in the directories and have a main rakefile load them.
The downside to this is that there is no namespaces in the Rakefile, all
tasks exist at a single global level, so there would need to be some
coordination between the subdirectories.

Have you thought of doing the equivalent of Jambase and having a packaged
library of default rules that will be applied, based on the platform that
Rake is invoked on? So, if Linux, CC=gcc, if Win32 CC=cl, etc..

Yep. I'm thinking through some ways of making namespaces work with Rake tasks
and how library modules might fit into this. When RubyGems slows down a bit,
I plan to revisit Rake and do something along the lines that you suggest.

Thanks for the feedback.

 

[Mark Probert]

Hi ..

Likewise, a pattern based rule might work here too (again, untested):

rule(/^test\d+$/, '.cpp') do |t|
sh "#{CC} #{CCOPT} -o #{t.name} #{t.prerequisites.join(' ')}"
end

Works well. My rule looks like:

rule( '.exe' => [
proc {|tn| tn.sub(/\.[^.]+$/, '.cpp').insert(0, "#{UT_DIR}\/") }
]) do |t|
bn = t.name.sub("\.exe", "")
objlist = check_dep(UT_DIR + "/" + bn + ".h")
sh( "#{CC} #{CCOPT} -o #{UT_DIR}/#{bn} #{t.prerequisites.join(' ')} \
#{objlist}" )
end

So, I use a synthetic target and use the check_dep() call to extract the
dependent .o file from the #include (thanks for the tip ).

And then the dependencies can be given explicitly.

file "test1" => ["test1.cpp", "foo.o"]

This also works, though I don't think that I have dependencies are quite
right.

In order to get things to work correctly, I have created a synthetic target
that enables an extension rule to be triggered. So:

file "ut_buffer" => [ "scanner.o", "ut_buffer.exe", ] do |t|
puts " .. running tests --> #{t.name}"
sh( "#{UT_DIR}/#{t.name}" )
end

So, this works well, though I would like not to have to specify the ".o"
dependency on the UT driver line. I assume that the way to do this is to add
the check_dep() to the proc and join the list to the prerequisites?

Regards,