Nested struct alignment summarized

[Robert Feldt]

Hi,

So trying to distill the recent thread about C compilers
alignment/layout of nested structs it appears that:

* Ruby/DL cannot as of now handle structs within structs.

* A trick to get around that is to inline the inner struct
into the outer struct.

* If we could assume this trick work in general we could
enhance Ruby/DL to handle nested structs.

* The trick relies on how different C compilers lays out
a struct in memory. The ANSI C standard does not specify
how that should be done; its implementation-defined ie.
each compiler must document how they do it but it may
not be in the same way.

* This means problem for us when trying to use Ruby/DL
for libs that have nested struct constructs.

So the situation look grim; even if we find a portable
way to check how a certain compiler lays things out
it may not lay things out in the same way for different
structs. There could be special situation where it changes the layout.

So is there anything we can do?

If not this sounds like a serious problem for using
Ruby/DL on larger/more complex lib "wraps".

Regards,

Robert Feldt

 

[Mauricio Fernández]

So the situation look grim; even if we find a portable
way to check how a certain compiler lays things out
it may not lay things out in the same way for different
structs. There could be special situation where it changes the layout.

So is there anything we can do?

Expand the current method (list of structure types in dl.h (ALIGN_*)),
so that the alignments for a growing number of structs are determined
at compile time?

Of course, that'd require some code to check for structural equivalence
of emm structures.

--
_ _
| |__ __ _| |_ ___ _ __ ___ __ _ _ __
| '_ \ / _` | __/ __| '_ ` _ \ / _` | '_ \
| |_) | (_| | |_\__ \ | | | | | (_| | | | |
|_.__/ \__,_|\__|___/_| |_| |_|\__,_|_| |_|
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

Yes I have a Machintosh, please don't scream at me.
-- Larry Blumette on linux-kernel

 

[Robert Feldt]

Expand the current method (list of structure types in dl.h (ALIGN_*)),
so that the alignments for a growing number of structs are determined
at compile time?

Of course, that'd require some code to check for structural equivalence
of emm structures.

Below is code to assist analysis of this. I'd be interested in the
output from the script on other machines and with other compilers. I've run it on a PC/WinXP/Cygwin

Ruby version: 1.8.0 (2003-09-08) [i386-cygwin]
Compiler: gcc
Version: gcc (GCC) 3.2 20020927 (prerelease)

and a Gentoo Linux machine

Ruby version: 1.8.0 (2003-08-04) [i686-linux-gnu]
Compiler: gcc
Version: gcc (GCC) 3.2.3 20030422 (Gentoo Linux 1.4 3.2.3-r1, propo
lice)

and results differ in how they align double's. They both inline
structs directly into the outer struct though.

Regards,

Robert

require 'dl/import'
require 'dl/struct'

your_name, your_mail = ARGV[0], ARGV[1]
compiler = ARGV[2] || "gcc"
compile_cmd = ARGV[3] || "-shared -o DLLNAME"
version_cmd = ARGV[4] || "--version"

class CFile
def initialize(compiler, compileCommand, versionCommand)
@co, @cc, @vc = compiler, compileCommand, versionCommand
@count = 0
end

def compile_to_dll(dllname = "test.so", code = "", cfilename = "test.c")
File.open(cfilename, "w") {|fh| fh.write code}
cmd = "#{@co} #{@cc} #{cfilename}".gsub("DLLNAME", dllname)
system cmd
File.delete cfilename
end

def compiler_version
`#{@co} #{@vc}`
end

Template = <<EOT
typedef struct I {
it1 i1;
it2 i2;
} I;

typedef struct O {
ot1 o1;
ot2 o2;
I i;
ot3 o3;
} O;

extern int o2o1() {O o; return ((int)&(o.o2) - (int)&o.o1);}
extern int i1o2() {O o; return ((int)&(o.i.i1) - (int)&o.o2);}
extern int i2i1() {O o; return ((int)&(o.i.i2) - (int)&o.i.i1);}
extern int o3i2() {O o; return ((int)&(o.o3) - (int)&o.i.i2);}
EOT

MethodNames = ["o2o1", "i1o2", "i2i1", "o3i2"]
TypeNames = ["it1", "it2", "ot1", "ot2", "ot3"]
DefaultTypes = Hash.new
TypeNames.each {|n| DefaultTypes[n] = "int"}

def address_diffs(typeAssignment = {})
dllname = new_dllname
compile_to_dll(dllname, create_cfile(typeAssignment))
load_and_get_adress_diffs(dllname)
end

def new_dllname
dllname = "test#{@count += 1}.so"
end

def create_cfile(typeAssignment)
ta = DefaultTypes.clone.update typeAssignment
cfile = Template.clone
ta.each {|n, t| cfile.gsub!(n, t)}
cfile
end

def load_and_get_adress_diffs(dllname)
dll = DL.dlopen("./" + dllname)
MethodNames.map {|mn| dll.sym(mn, 'I').call().first}
end
end

cf = CFile.new compiler, compile_cmd, version_cmd

types = ["char", "int", "short", "long", "void*", "float", "double"]

def all_pairs(ary)
a, na = ary.uniq, []
a.map {|t1| a.map {|t2| na << [t1, t2]}}
na
end

report = <<EOR
Reporter: #{your_name}
Email: #{your_mail}
Ruby version: #{RUBY_VERSION} (#{RUBY_RELEASE_DATE}) [#{RUBY_PLATFORM}]
Compiler: #{compiler}
Version: #{cf.compiler_version}
Build command: #{compiler} #{compile_cmd}
EOR

puts report

results, counts = [], Hash.new(0)
all_pairs(types).each do |tfrst, tsnd|
CFile::MethodNames.each do |methodname|
t1, t2 = methodname[0,2], methodname[2,2]
t1[1,0] = t2[1,0] = "t"
th = {t1 => tfrst, t2 => tsnd}
adiffs = cf.address_diffs(th)
results << ["#{t1} #{tfrst}, #{t2} #{tsnd}:", adiffs]
counts[adiffs] += 1
STDERR.print "."; STDERR.flush
end
end

most_common = counts.to_a.sort {|a,b| b.last <=> a.last}.first.first

puts "The address diffs are #{most_common.inspect} for all combinations but:"

# Print the uncommon ones
results.each {|r| puts "#{r[0]} #{r[1].inspect}" unless r[1] == most_common}