32/64 assembly question

I am wondering whether I can get gcc or as (or any other compiler or
assembler) to compile a C program in ILP32 model, but to use amd64
instructions that will work even in the 64-bit/long mode mode. The
default operand size when in long mode is 32 bits. The default address
size is 64 bits but, at least in theory, the address size can be
overridden using an address size override prefix.

To make my point clearer here the steps

Step 1 write t.c

void xcopy(char *s, char *d, int n)
{
while (n) {
*d++ = *s++;
n--;
}
return;
}

Step 2 Compile t.c into assembly code in ILP32 mode

gcc -m32 -O -S t.c

Step 3 Modify t.s as mentioned below

.file "t.c"
.text
.p2align 4,,15
..globl xcopy
.type xcopy, @function
xcopy:
.code64 ====== manually added
pushl %ebp
movl %esp, %ebp
pushl %ebx
movl 8(%ebp), %ebx
movl 12(%ebp), %ecx
movl 16(%ebp), %edx
testl %edx, %edx
je .L6
.p2align 4,,7
..L4:
movzbl (%ebx), %eax
incl %ebx
movb %al, (%ecx)
incl %ecx
decl %edx
jne .L4
..L6:
popl %ebx
leave
ret
.size xcopy, .-xcopy
.ident "GCC: (GNU) 3.4.6 [FreeBSD] 20060305"


Step 4 compile t.s in the 64 bit mode
[root@pi_mpx ~/pk]# gcc -m64 t.s
t.s: Assembler messages:
t.s:7: Error: suffix or operands invalid for `push'
t.s:9: Error: suffix or operands invalid for `push'
t.s:10: Error: `8(%ebp)' is not a valid 64 bit base/index expression
t.s:11: Error: `12(%ebp)' is not a valid 64 bit base/index expression
t.s:12: Error: `16(%ebp)' is not a valid 64 bit base/index expression
t.s:17: Error: `(%ebx)' is not a valid 64 bit base/index expression
t.s:19: Error: `(%ecx)' is not a valid 64 bit base/index expression
t.s:24: Error: suffix or operands invalid for `pop'

Obviously it didn't work. But if there was a way to tell gcc to select
instructions carefully such that they will be valid both in
compatibility and long mode ... then I would have had a ILP32 program
running in 64 bit mode.

Can this be made to work?

Regards,
Prakash

On Dec 4, 1:27*am, pkhemani wrote:
I am wondering whether I can get gcc or as (or any other compiler or
assembler) to compile a C program in ILP32 model, but to use amd64
instructions that will work even in the 64-bit/long mode mode. The
default operand size when in long mode is 32 bits. The default address
size is 64 bits but, at least in theory, the address size can be
overridden using an address size override prefix.

To make my point clearer here the steps

Step 1 *write t.c

void xcopy(char *s, char *d, int n)
{
* * * * while (n) {
* * * * * * * * *d++ = *s++;
* * * * * * * * n--;
* * * * }
* * * * return;

}

Step 2 Compile t.c into assembly code in ILP32 mode

gcc -m32 -O -S t.c

Step 3 Modify t.s as mentioned below

* * * * .file * "t.c"
* * * * .text
* * * * .p2align 4,,15
.globl xcopy
* * * * .type * xcopy, @function
xcopy:
* * * * .code64 * * * * * * *====== manually added
* * * * pushl * %ebp
* * * * movl * *%esp, %ebp
* * * * pushl * %ebx
* * * * movl * *8(%ebp), %ebx
* * * * movl * *12(%ebp), %ecx
* * * * movl * *16(%ebp), %edx
* * * * testl * %edx, %edx
* * * * je * * *.L6
* * * * .p2align 4,,7
.L4:
* * * * movzbl *(%ebx), %eax
* * * * incl * *%ebx
* * * * movb * *%al, (%ecx)
* * * * incl * *%ecx
* * * * decl * *%edx
* * * * jne * * .L4
.L6:
* * * * popl * *%ebx
* * * * leave
* * * * ret
* * * * .size * xcopy, .-xcopy
* * * * .ident *"GCC: (GNU) 3.4.6 [FreeBSD] 20060305"

Step 4 compile t.s in the 64 bit mode
[root@pi_mpx ~/pk]# gcc -m64 t.s
t.s: Assembler messages:
t.s:7: Error: suffix or operands invalid for `push'
t.s:9: Error: suffix or operands invalid for `push'
t.s:10: Error: `8(%ebp)' is not a valid 64 bit base/index expression
t.s:11: Error: `12(%ebp)' is not a valid 64 bit base/index expression
t.s:12: Error: `16(%ebp)' is not a valid 64 bit base/index expression
t.s:17: Error: `(%ebx)' is not a valid 64 bit base/index expression
t.s:19: Error: `(%ecx)' is not a valid 64 bit base/index expression
t.s:24: Error: suffix or operands invalid for `pop'

Obviously it didn't work. But if there was a way to tell gcc to select
instructions carefully such that they will be valid both in
compatibility and long mode ... then I would have had a ILP32 program
running in 64 bit mode.

Can this be made to work?

Regards,
Prakash

Can you elaborate on what exactly you're trying to achieve?

A few points...

0. In 64-bit assembler you can't do "movb %al, (%ecx)" since there's
no such addressing mode, at least formally. It has to be "movb %al,
(%rcx)" with the appropriate value of rcx. This can be prefixed with
the address override prefix, though.
1. Some instructions in 64-bit mode have 64-bit default operand (e.g.
stack-related instructions) and there's no way to override them to 32
bits (16 possible, though). See the AMD documentation for details.
You'll need to do something about the PUSH/POP instructions (replace
them with something that functions as PUSH/POP with 32-bit operands).
2. You can use 32-bit pointers in 64-bit mode if and only if all the
code and data (including stack) of the application is below 4G virtual
address (you just truncate/ignore the top 32 bits of the address) OR
is within the same 4G (that is, the top 32 bits of all addresses are
the same; here you modify the code in such a way that these top 32
bits are a known constant part of all pointers and the low 32 bits are
variable, kind of like the default segments in a .COM program allowing
it to access 64K just like that).
3. Some instructions are unavailable in 64-bit code. See the docs.
4. One address encoding isn't available in 64-bit mode (due to RIP-
relative addressing).
5. Normally, the same binary can't run in 32-bit and 64-bit
environments and produce the same results. You need to restrict the
instruction set used and replace some instructions by others to be
able to do that in some way.
6. Expect issues with the code, tools and operating environment.

Alex

In comp.lang.asm.x86 pkhemani wrote:
I am wondering whether I can get gcc or as (or any other compiler or
assembler) to compile a C program in ILP32 model, but to use amd64
instructions that will work even in the 64-bit/long mode mode. The
default operand size when in long mode is 32 bits. The default address
size is 64 bits but, at least in theory, the address size can be
overridden using an address size override prefix.

To make my point clearer here the steps

Step 1 write t.c

Step 2 Compile t.c into assembly code in ILP32 mode

gcc -m32 -O -S t.c

Step 3 Modify t.s as mentioned below

Step 4 compile t.s in the 64 bit mode

t.s:7: Error: suffix or operands invalid for `push'

Obviously it didn't work. But if there was a way to tell gcc to select
instructions carefully such that they will be valid both in
compatibility and long mode ... then I would have had a ILP32 program
running in 64 bit mode.

Can this be made to work?

It is not clear what you want to do. Do you want to have the
same binary working both in 32 and 64 bit mode? Then you
will have problems with system calls and program header (they
differ between 32 and 64-bit mode). If you want just generate
program which uses 32-bit model and tolerate small amount of
64-bit code than you can write apropriate asm by had. Note
that you need to replace pop/push by mov instruction. If
you insist on 32-bit return adresses you have to replace
call/ret by jumps. So it is technically possible.

If you want re-use gcc to generate such code than I think
you need to port gcc to new model -- AFAIK such thing was
done for MIPS (n32 model) but there are no plans to do it
for AMD64. Simply slapping .code 64 before 32-bit instructions
is not going to work and changes look to deep to do them
by postprocessing assembly output.

Also to make you code usable you probably need to port C
library and possibly some other libraries -- looks like
substantial work.

Coming back to your question, it is not clear why do you
want ILP32 code running in 64-bit mode. Normally you
can run both 32-bit and 64-bit code on the same processor,
even in single address space. If you want to call
32-bit code from 64-bit or vice versa you will have bigger
problems than mode switching -- you need a set of routines
to convert parameters. If you think that ILP32 code
in 64-bit mode will give you better performance than you
probably want compiler port to use extra registers.
Lacking compiler port it is probably easier to re-write
program to use 32-bit data types, compile it in 64-bit
mode make sure all data is in first 4G of memory.

If you want ILP 32 for compatibility reasons but do not
care much about performance than you may try adapt Qemu
so that Qemu runs in 64-bit mode and executes 32-bit
binary of your program. Alternatively you may adapt
some simple C compiler (porting non-optimizing compiler
is easier than optimizing one like gcc).

--
Waldek Hebisch

Thank you all for your replies. Here is some background.

I have ILP32 code that doesn't use the C library and doesn't make any
system calls.

Parts of this code needs larger address space. It is difficult to port
the entire program to 64 bits. But we can port parts of it to 64 bit,
massage the x86 32-bit ABI into amd64 ABI, and make far-jumps between
32 bit and 64 bit code.

The issue is that far-jumps are expensive - 20 times or more
expensive. (I am not too sure whether all far jumps are expensive or
only the ones that switch between compatibility and long mode are
expensive.) SYSCALL/SYSRET don't work in 32 bit mode on Intel CPUs
(they do in AMD). I am currently experimenting with SYSENTER/SYSEXIT
but from what I have read it will not perform as well. (SYSENTER will
also change the privelege level ... but if it is fast enough then I
can work around that).

So this is where my question comes in. What if I could always stay in
long mode, execute my ILP32 code in long mode and make near-jumps to
64-bit code when I have to?

From what I have read so far, theoretically it is possible but there
isn't much incentive for the tools to support such a thing.

Thanks,
Prakash




On Dec 3, 5:15*pm, "Alexei A. Frounze" wrote:
On Dec 4, 1:27*am, pkhemani wrote:



I am wondering whether I can get gcc or as (or any other compiler or
assembler) to compile a C program in ILP32 model, but to use amd64
instructions that will work even in the 64-bit/long mode mode. The
default operand size when in long mode is 32 bits. The default address
size is 64 bits but, at least in theory, the address size can be
overridden using an address size override prefix.

To make my point clearer here the steps

Step 1 *write t.c

void xcopy(char *s, char *d, int n)
{
* * * * while (n) {
* * * * * * * * *d++ = *s++;
* * * * * * * * n--;
* * * * }
* * * * return;

}

Step 2 Compile t.c into assembly code in ILP32 mode

gcc -m32 -O -S t.c

Step 3 Modify t.s as mentioned below

* * * * .file * "t.c"
* * * * .text
* * * * .p2align 4,,15
.globl xcopy
* * * * .type * xcopy, @function
xcopy:
* * * * .code64 * * * * * * *====== manually added
* * * * pushl * %ebp
* * * * movl * *%esp, %ebp
* * * * pushl * %ebx
* * * * movl * *8(%ebp), %ebx
* * * * movl * *12(%ebp), %ecx
* * * * movl * *16(%ebp), %edx
* * * * testl * %edx, %edx
* * * * je * * *.L6
* * * * .p2align 4,,7
.L4:
* * * * movzbl *(%ebx), %eax
* * * * incl * *%ebx
* * * * movb * *%al, (%ecx)
* * * * incl * *%ecx
* * * * decl * *%edx
* * * * jne * * .L4
.L6:
* * * * popl * *%ebx
* * * * leave
* * * * ret
* * * * .size * xcopy, .-xcopy
* * * * .ident *"GCC: (GNU) 3.4.6 [FreeBSD] 20060305"

Step 4 compile t.s in the 64 bit mode
[root@pi_mpx ~/pk]# gcc -m64 t.s
t.s: Assembler messages:
t.s:7: Error: suffix or operands invalid for `push'
t.s:9: Error: suffix or operands invalid for `push'
t.s:10: Error: `8(%ebp)' is not a valid 64 bit base/index expression
t.s:11: Error: `12(%ebp)' is not a valid 64 bit base/index expression
t.s:12: Error: `16(%ebp)' is not a valid 64 bit base/index expression
t.s:17: Error: `(%ebx)' is not a valid 64 bit base/index expression
t.s:19: Error: `(%ecx)' is not a valid 64 bit base/index expression
t.s:24: Error: suffix or operands invalid for `pop'

Obviously it didn't work. But if there was a way to tell gcc to select
instructions carefully such that they will be valid both in
compatibility and long mode ... then I would have had a ILP32 program
running in 64 bit mode.

Can this be made to work?

Regards,
Prakash

Can you elaborate on what exactly you're trying to achieve?

A few points...

0. In 64-bit assembler you can't do "movb %al, (%ecx)" since there's
no such addressing mode, at least formally. It has to be "movb %al,
(%rcx)" with the appropriate value of rcx. This can be prefixed with
the address override prefix, though.
1. Some instructions in 64-bit mode have 64-bit default operand (e.g.
stack-related instructions) and there's no way to override them to 32
bits (16 possible, though). See the AMD documentation for details.
You'll need to do something about the PUSH/POP instructions (replace
them with something that functions as PUSH/POP with 32-bit operands).
2. You can use 32-bit pointers in 64-bit mode if and only if all the
code and data (including stack) of the application is below 4G virtual
address (you just truncate/ignore the top 32 bits of the address) OR
is within the same 4G (that is, the top 32 bits of all addresses are
the same; here you modify the code in such a way that these top 32
bits are a known constant part of all pointers and the low 32 bits are
variable, kind of like the default segments in a .COM program allowing
it to access 64K just like that).
3. Some instructions are unavailable in 64-bit code. See the docs.
4. One address encoding isn't available in 64-bit mode (due to RIP-
relative addressing).
5. Normally, the same binary can't run in 32-bit and 64-bit
environments and produce the same results. You need to restrict the
instruction set used and replace some instructions by others to be
able to do that in some way.
6. Expect issues with the code, tools and operating environment.

Alex

Alexei A. Frounze wrote:
0. In 64-bit assembler you can't do "movb %al, (%ecx)" since there's
no such addressing mode, at least formally. It has to be "movb %al,
(%rcx)" with the appropriate value of rcx. This can be prefixed with
the address override prefix, though.

It's perfectly formal, but gas doesn't understand it. It's a gas bug.
gas only recently learned to deal with address size prefixes for 16 and
32 bit mode, so it's not all that surprising.

4. One address encoding isn't available in 64-bit mode (due to RIP-
relative addressing).

Shouldn't matter, though, since there is an equivalent encoding.

5. Normally, the same binary can't run in 32-bit and 64-bit
environments and produce the same results. You need to restrict the
instruction set used and replace some instructions by others to be
able to do that in some way.

If you want to run a 32-bit binary directly, just run it in
compatibility mode. If what you need is a polyglot *binary* (one that
can be run in either mode), the easiest way is probably to simply detect
your mode and then jump to either 32- or 64-bit code.

Here is a stub which will run in either mode and do just that:

1 bits 32
2
3 00000000 31C0 xor eax,eax
4 00000002 40 inc eax
5 00000003 90 nop
6 00000004 0F8401000000 jz near code64
7
8 code32:
[...]
11 code64:

In 64-bit mode, this turns into:

00000000 31C0 xor eax,eax
00000002 4090 nop
00000004 0F8401000000 jz dword 0xb

-hpa

pkhemani wrote:
Parts of this code needs larger address space. It is difficult to port
the entire program to 64 bits. But we can port parts of it to 64 bit,
massage the x86 32-bit ABI into amd64 ABI, and make far-jumps between
32 bit and 64 bit code.

The issue is that far-jumps are expensive - 20 times or more
expensive.

I don't think the concept of a far jump exists anymore in 64-bit mode.
That is if the concept of a far jump is a "segmentffset" type of jump.
All jumps in 64-bit mode should be near. It ignores the segment
overrides in 64-bit mode.

Yousuf Khan

Yousuf Khan wrote:
pkhemani wrote:
Parts of this code needs larger address space. It is difficult to port
the entire program to 64 bits. But we can port parts of it to 64 bit,
massage the x86 32-bit ABI into amd64 ABI, and make far-jumps between
32 bit and 64 bit code.

The issue is that far-jumps are expensive - 20 times or more
expensive.

I don't think the concept of a far jump exists anymore in 64-bit mode.
That is if the concept of a far jump is a "segmentffset" type of jump.
All jumps in 64-bit mode should be near. It ignores the segment
overrides in 64-bit mode.


No, far jumps still exist in 64-bit mode. Code segment transfers still
matter in 64-bit mode, since they are the mechanism to switch between
kernel and user mode as well as between 64-bit mode and compatibility mode.

-hpa

On Dec 3 2008, 4:27 pm, pkhemani wrote:
I am wondering whether I can get gcc or as (or any other compiler or
assembler) to compile a C program in ILP32 model, but to use amd64
instructions that will work even in the 64-bit/long mode mode. The
default operand size when in long mode is 32 bits. The default address
size is 64 bits but, at least in theory, the address size can be
overridden using an address size override prefix.

I think the responders have forgotten that the larger address
space is not the only benefit of 64-bit mode.
If one hasn't more than 4GB of memory, the larger address space is a
detriment:
Pointers take twice as much memory as needed.
If one uses a lot of pointers, that could be too much.
It would be nice to have 32-bit pointers and keep the other benefits.

To make my point clearer here the steps

Step 1 write t.c

void xcopy(char *s, char *d, int n)
{
while (n) {
*d++ = *s++;
n--;
}
return;

}

In C++:

templaceclass Target class ShortPtr {
public:
typedef Target *Ptr; // 64 bits

ShortPtr(Ptr p)
: datum(static_castunsigned(reinterpret_castunsig ned long
(p))) {}
operator Ptr() const
{ return reinterpret_castPtr(static_castunsigned long
(datum)); }
private:
unsigned datum; // 32 bits
} // ShortPtr

// This probably wreaks hell with optimization.
// 'Twould be nice if the compiler did it natively.

// Did I get my casts right?

void xcopy(ShortPtrchars, ShortPtrchard, int n)
{
// as before
....
} // xcopy

Can an x-64 do a 32- to 64-bit load?
A 64- to 32-bit store?
IIRC gcc has an attribute or compiler option
that affects the size of long doubles.
Is there a reason something similar couldn't or
shouldn't be done with pointers on 64-bit machines?