Compare commits

...

30 Commits

Author SHA1 Message Date
Roland Paterson-Jones
18f6f2d5ea Simple Inner Loop Optimzation
Two simple loop optimizations.

1. Strength reduction of mul[tiplication] by loop induction
variable.

2. Hoisting of (address) base into phi where loop induction
variable is used only as a base (address) offset.

Limited to loops with a single body block, which happily
is always innermost loops. This restriction would not be
very hard to lift - it would require detecting the set of
loop blocks (and ensuring reducibility?)

Limited to loop induction variables with 0 initial value
and increment of 1 (for mul strength reduction). This
limitation is trivial to lift; however all of the
cproc/hare[c]/coremark opportunity is with 0/1 loops for
mul reduction, and 0 initial value for base-offset opt.
2026-05-03 14:37:27 +02:00
Quentin Carbonneaux
ac48f83f17 extern DYNCONST
New DYNCONST flag to access symbols from
dynamically-linked libraries. It can also
be used by frontends to implement PIC.

Review by mcf
2026-05-03 11:34:53 +02:00
Quentin Carbonneaux
577b4667db negated conditions rework
Floating point comparisons are subtle due to
unordered operands (i.e., when one or two of
the operands are nans).

In amd64 we arrange post isel to make sure we
only have comparisons that will return false
on unordered operands and for which we have a
negated version (returning true on unordered
operands).  I made this situation a bit more
explicit by marking unexpected comparisons
with "?" in amd64/emit.c.

In arm64, the instruction set is rich enough
to have a negated version for all operators
that returns true on unordered inputs. So we
build a backend specific table.

I made sure to remove cmpneg() from utils.c;
it is a footgun as it does not work well for
floating point ops because of the 'unordered'
edge case.

I found the ARM docs about condition codes
pretty bad; the following C program is a
good substitute:

    #include <stdio.h>
    int
    flags(float a, float b)
    {
    	int z, c, n, v, le;
    	__asm__(
    		"fcmpe %s5, %s6\n"
    		"\tcset %0, eq\n"
    		"\tcset %1, mi\n"
    		"\tcset %2, cs\n"
    		"\tcset %3, vs\n"
    		"\tcset %4, le\n"
    	: "=r"(z),"=r"(n),"=r"(c),"=r"(v),"=r"(le)
    	: "x"(a), "x"(b)
    	: "cc");
    	printf(
    		"cmp(%g,%g): z=%d n=%d c=%d v=%d le=%d\n",
    		a, b, z, n, c, v, le);
    }
    int
    main()
    {
    	flags(1.0, 1.0);
    	flags(0.0, 1.0);
    	flags(1.0, 0.0);
    	flags(0.0, 0.0/0.0);
    	flags(0.0/0.0, 0.0);
    }

Compile & run with:

    aarch64-linux-gnu-gcc -static -no-pie flags.c
    qemu-aarch64 ./a.out
2026-04-29 17:45:20 +02:00
Quentin Carbonneaux
bef981e9a1 no cmpneg() for float comparisons 2026-04-29 15:04:27 +02:00
Quentin Carbonneaux
0454fa259b arm64: fix unordered fp comparisons 2026-04-29 13:37:22 +02:00
Michael Forney
dba8d5a4bf update debug flag list for gvn/gcm 2026-04-27 10:08:45 +02:00
Michael Forney
504a2012f4 fix float neg on mach-o
neg for float uses xorp[sd] with a 16-byte memory operand.

This matches what clang emits with --target=x86_64-apple-darwin.
2026-04-25 18:51:46 +02:00
Quentin Carbonneaux
b58e2e695b fix exponential complexity in usewidthle() 2026-04-21 13:25:00 +02:00
willow
8ff0651552 parse: deny non-digit after minus in getint
also: '0' <= c <= '9' -> isdigit(c)
Signed-off-by: willow <im@purring.fyi>
2026-02-28 19:05:57 +01:00
willow
7ac9722ccb remove unused variable
Signed-off-by: willow <im@purring.fyi>
2026-02-28 19:05:15 +01:00
Quentin Carbonneaux
4f9b94a9b3 formatting fixes in tests 2026-02-12 09:23:13 +01:00
Quentin Carbonneaux
5f40188f9e cosmetics in emit.c 2026-02-12 09:21:15 +01:00
Scott Graham
01102ad63b winabi: fix isel of large consts
ABI lowering for winabi was incorrectly using a Kl cls when emitting
Ostorel, which in turn was causing isel to fail to lower large constants
in to temporaries ( https://c9x.me/git/qbe.git/tree/amd64/isel.c#n107 )
when necessary.

(This should be applied on the 'winabi' branch.)
2026-02-12 09:17:17 +01:00
Scott Graham
d5f02dc67c winabi: fix allocation of parameters to regs with hidden arg
In the presence of the hidden arg for return-by-value, the
registers used for natural arguments were incorrect.

(This should be applied on the 'winabi' branch.)
2026-02-12 09:17:17 +01:00
Scott Graham
d166a61141 Implementation of Windows amd64_win target
This is an implementation of the Windows ABI. It supports most features
(struct passing/returning, varargs, env). TLS is not yet supported.

This patch does not actually port QBE to Windows, it only allows QBE to
generate correct asm to target Windows. As a result, testing is
accomplished on a Linux host, by using a cross-compiling toolchain, and
running the resulting binaries by using wine. See:

	TARGET=amd64_win tools/test.sh all

A few cross-platform tests were changed from 'long' to 'long long' in
driver code because long in C does not match the size of a QBE 'l' on
Windows.
2026-02-12 09:17:17 +01:00
Richard McCormack
cf06ce159d Modify amd64 fixarg to fix calling constant addresses
On x86_64, direct calls are always PC-relative. This means that
in order to call an absolute address, the call must be indirect.

To accomplish this, update fixarg to introduce a temporary before
emitting.
2026-01-13 21:24:31 +01:00
Quentin Carbonneaux
640c78d0da fix typo in simplcfg 2026-01-13 20:36:23 +01:00
Quentin Carbonneaux
afd5d2e518 drop dead preds in fixphis
It is possible that GVN removes
some dead blocks, this could lead
to odd - but probably harmless -
phi args appearing in the IL.
This patch cleans things up during
fillcfg().
2026-01-13 18:27:50 +01:00
Quentin Carbonneaux
e8365dd0a2 new simplcfg pass
Useful for ifopt to match more
often. Empty blocks are fused
and conditional jumps on empty
blocks with the same successor
(and no phis in the successor)
are collapsed.
2026-01-13 18:17:35 +01:00
Quentin Carbonneaux
c6336557da ifopt simplifications 2026-01-13 18:11:37 +01:00
Roland Paterson-Jones
5c1eb24e2c If-conversion RFC 4 - x86 only (for now), use cmovXX
Replacement of tiny conditional jump graphlets with
conditional move instructions.

Currently enabled only for x86. Arm64 support using cselXX
will be essentially identical.

Adds (internal) frontend sel0/sel1 ops with flag-specific
backend xselXX following jnz implementation pattern.

Testing: standard QBE, cproc, harec, hare, roland
2026-01-13 18:11:30 +01:00
Quentin Carbonneaux
7201079137 update copyright years 2026-01-13 17:09:41 +01:00
Quentin Carbonneaux
112cc1b824 rv64: handle slots in jnz 2026-01-13 17:09:41 +01:00
Quentin Carbonneaux
6a2dca8b99 fix jmp arg spilling
In case we need to spill to accomodate
for the jump argument, piggyback the
reloads from slots to regalloc so that
they can be correctly inserted on edges.
2026-01-13 17:09:41 +01:00
Quentin Carbonneaux
e0ded59639 please as with truncated constants
Apple's assembler actually hard
crashed on overflows.
2026-01-06 20:43:42 +01:00
Quentin Carbonneaux
0f6bbb1c7c arm64_apple: fix argxbh support 2026-01-06 20:43:42 +01:00
Quentin Carbonneaux
73f0accb45 arm64: prevent bogus IP1 clobbers 2026-01-06 20:43:38 +01:00
Quentin Carbonneaux
03da40271f rv64: fix invalid float immediates
Thanks to Luke Graham for reporting
and fixing this issue.
2026-01-05 22:19:53 +01:00
Quentin Carbonneaux
120f316162 skip deleted phis in use width scan 2025-05-30 17:40:17 +02:00
Quentin Carbonneaux
8d5b86ac4c fix fp constants on big endian hosts 2025-04-16 10:29:00 +02:00
40 changed files with 2412 additions and 302 deletions

View File

@ -1,4 +1,4 @@
© 2015-2025 Quentin Carbonneaux <quentin@c9x.me> © 2015-2026 Quentin Carbonneaux <quentin@c9x.me>
Permission is hereby granted, free of charge, to any person obtaining a Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"), copy of this software and associated documentation files (the "Software"),

View File

@ -5,8 +5,9 @@ PREFIX = /usr/local
BINDIR = $(PREFIX)/bin BINDIR = $(PREFIX)/bin
COMMOBJ = main.o util.o parse.o abi.o cfg.o mem.o ssa.o alias.o load.o \ COMMOBJ = main.o util.o parse.o abi.o cfg.o mem.o ssa.o alias.o load.o \
copy.o fold.o gvn.o gcm.o simpl.o live.o spill.o rega.o emit.o copy.o fold.o gvn.o gcm.o loopopt.o simpl.o ifopt.o live.o \
AMD64OBJ = amd64/targ.o amd64/sysv.o amd64/isel.o amd64/emit.o spill.o rega.o emit.o
AMD64OBJ = amd64/targ.o amd64/sysv.o amd64/isel.o amd64/emit.o amd64/winabi.o
ARM64OBJ = arm64/targ.o arm64/abi.o arm64/isel.o arm64/emit.o ARM64OBJ = arm64/targ.o arm64/abi.o arm64/isel.o arm64/emit.o
RV64OBJ = rv64/targ.o rv64/abi.o rv64/isel.o rv64/emit.o RV64OBJ = rv64/targ.o rv64/abi.o rv64/isel.o rv64/emit.o
OBJ = $(COMMOBJ) $(AMD64OBJ) $(ARM64OBJ) $(RV64OBJ) OBJ = $(COMMOBJ) $(AMD64OBJ) $(ARM64OBJ) $(RV64OBJ)
@ -80,6 +81,9 @@ check-arm64: qbe
check-rv64: qbe check-rv64: qbe
TARGET=rv64 tools/test.sh all TARGET=rv64 tools/test.sh all
check-amd64_win: qbe
TARGET=amd64_win tools/test.sh all
src: src:
@echo $(SRCALL) @echo $(SRCALL)

25
all.h
View File

@ -44,6 +44,7 @@ enum {
struct Target { struct Target {
char name[16]; char name[16];
char apple; char apple;
char windows;
int gpr0; /* first general purpose reg */ int gpr0; /* first general purpose reg */
int ngpr; int ngpr;
int fpr0; /* first floating point reg */ int fpr0; /* first floating point reg */
@ -62,6 +63,7 @@ struct Target {
void (*emitfin)(FILE *); void (*emitfin)(FILE *);
char asloc[4]; char asloc[4];
char assym[4]; char assym[4];
uint cansel:1;
}; };
#define BIT(n) ((bits)1 << (n)) #define BIT(n) ((bits)1 << (n))
@ -183,6 +185,8 @@ enum {
Oalloc1 = Oalloc16, Oalloc1 = Oalloc16,
Oflag = Oflagieq, Oflag = Oflagieq,
Oflag1 = Oflagfuo, Oflag1 = Oflagfuo,
Oxsel = Oxselieq,
Oxsel1 = Oxselfuo,
NPubOp = Onop, NPubOp = Onop,
Jjf = Jjfieq, Jjf = Jjfieq,
Jjf1 = Jjffuo, Jjf1 = Jjffuo,
@ -199,6 +203,7 @@ enum {
#define isparbh(o) INRANGE(o, Oparsb, Oparuh) #define isparbh(o) INRANGE(o, Oparsb, Oparuh)
#define isargbh(o) INRANGE(o, Oargsb, Oarguh) #define isargbh(o) INRANGE(o, Oargsb, Oarguh)
#define isretbh(j) INRANGE(j, Jretsb, Jretuh) #define isretbh(j) INRANGE(j, Jretsb, Jretuh)
#define isxsel(o) INRANGE(o, Oxsel, Oxsel1)
enum { enum {
Kx = -1, /* "top" class (see usecheck() and clsmerge()) */ Kx = -1, /* "top" class (see usecheck() and clsmerge()) */
@ -288,8 +293,10 @@ struct Use {
struct Sym { struct Sym {
enum { enum {
SGlo, SGlo = 0, /* direct access */
SThr, SThr = 1, /* local-exec TLS */
SExt = 2, /* GOT/PLT access */
SExtThr = SExt|SThr, /* initial-exec TLS */
} type; } type;
uint32_t id; uint32_t id;
}; };
@ -482,7 +489,7 @@ void *vnew(ulong, size_t, Pool);
void vfree(void *); void vfree(void *);
void vgrow(void *, ulong); void vgrow(void *, ulong);
void addins(Ins **, uint *, Ins *); void addins(Ins **, uint *, Ins *);
void addbins(Blk *, Ins **, uint *); void addbins(Ins **, uint *, Blk *);
void strf(char[NString], char *, ...); void strf(char[NString], char *, ...);
uint32_t intern(char *); uint32_t intern(char *);
char *str(uint32_t); char *str(uint32_t);
@ -495,7 +502,6 @@ void emiti(Ins);
void idup(Blk *, Ins *, ulong); void idup(Blk *, Ins *, ulong);
Ins *icpy(Ins *, Ins *, ulong); Ins *icpy(Ins *, Ins *, ulong);
int cmpop(int); int cmpop(int);
int cmpneg(int);
int cmpwlneg(int); int cmpwlneg(int);
int clsmerge(short *, short); int clsmerge(short *, short);
int phicls(int, Tmp *); int phicls(int, Tmp *);
@ -555,6 +561,8 @@ void fillloop(Fn *);
void simpljmp(Fn *); void simpljmp(Fn *);
int reaches(Fn *, Blk *, Blk *); int reaches(Fn *, Blk *, Blk *);
int reachesnotvia(Fn *, Blk *, Blk *, Blk *); int reachesnotvia(Fn *, Blk *, Blk *, Blk *);
int ifgraph(Blk *, Blk **, Blk **, Blk **);
void simplcfg(Fn *);
/* mem.c */ /* mem.c */
void promote(Fn *); void promote(Fn *);
@ -595,6 +603,12 @@ void gvn(Fn *);
int pinned(Ins *); int pinned(Ins *);
void gcm(Fn *); void gcm(Fn *);
/* ifopt.c */
void ifconvert(Fn *fn);
/* loopopt.c */
void loopopt(Fn *fn);
/* simpl.c */ /* simpl.c */
void simpl(Fn *); void simpl(Fn *);
@ -614,7 +628,8 @@ void emitfnlnk(char *, Lnk *, FILE *);
void emitdat(Dat *, FILE *); void emitdat(Dat *, FILE *);
void emitdbgfile(char *, FILE *); void emitdbgfile(char *, FILE *);
void emitdbgloc(uint, uint, FILE *); void emitdbgloc(uint, uint, FILE *);
int stashbits(void *, int); int stashbits(bits, int);
void elf_emitfnfin(char *, FILE *); void elf_emitfnfin(char *, FILE *);
void elf_emitfin(FILE *); void elf_emitfin(FILE *);
void macho_emitfin(FILE *); void macho_emitfin(FILE *);
void pe_emitfin(FILE *);

View File

@ -4,14 +4,14 @@ typedef struct Amd64Op Amd64Op;
enum Amd64Reg { enum Amd64Reg {
RAX = RXX+1, /* caller-save */ RAX = RXX+1, /* caller-save */
RCX, RCX, /* caller-save */
RDX, RDX, /* caller-save */
RSI, RSI, /* caller-save on sysv, callee-save on win */
RDI, RDI, /* caller-save on sysv, callee-save on win */
R8, R8, /* caller-save */
R9, R9, /* caller-save */
R10, R10, /* caller-save */
R11, R11, /* caller-save */
RBX, /* callee-save */ RBX, /* callee-save */
R12, R12,
@ -41,9 +41,13 @@ enum Amd64Reg {
NFPR = XMM14 - XMM0 + 1, /* reserve XMM15 */ NFPR = XMM14 - XMM0 + 1, /* reserve XMM15 */
NGPR = RSP - RAX + 1, NGPR = RSP - RAX + 1,
NGPS = R11 - RAX + 1,
NFPS = NFPR, NFPS = NFPR,
NCLR = R15 - RBX + 1,
NGPS_SYSV = R11 - RAX + 1,
NCLR_SYSV = R15 - RBX + 1,
NGPS_WIN = R11 - RAX + 1 - 2, /* -2 for RDI/RDI */
NCLR_WIN = R15 - RBX + 1 + 2, /* +2 for RDI/RDI */
}; };
MAKESURE(reg_not_tmp, XMM15 < (int)Tmp0); MAKESURE(reg_not_tmp, XMM15 < (int)Tmp0);
@ -63,8 +67,16 @@ bits amd64_sysv_retregs(Ref, int[2]);
bits amd64_sysv_argregs(Ref, int[2]); bits amd64_sysv_argregs(Ref, int[2]);
void amd64_sysv_abi(Fn *); void amd64_sysv_abi(Fn *);
/* winabi.c */
extern int amd64_winabi_rsave[];
extern int amd64_winabi_rclob[];
bits amd64_winabi_retregs(Ref, int[2]);
bits amd64_winabi_argregs(Ref, int[2]);
void amd64_winabi_abi(Fn *);
/* isel.c */ /* isel.c */
void amd64_isel(Fn *); void amd64_isel(Fn *);
/* emit.c */ /* emit.c */
void amd64_emitfn(Fn *, FILE *); void amd64_sysv_emitfn(Fn *, FILE *);
void amd64_winabi_emitfn(Fn *, FILE *);

View File

@ -12,24 +12,22 @@ struct E {
}; };
#define CMP(X) \ #define CMP(X) \
X(Ciule, "be") \ X(Ciule, "be", "a") \
X(Ciult, "b") \ X(Ciult, "b", "ae") \
X(Cisle, "le") \ X(Cisle, "le", "g") \
X(Cislt, "l") \ X(Cislt, "l", "ge") \
X(Cisgt, "g") \ X(Cisgt, "g", "le") \
X(Cisge, "ge") \ X(Cisge, "ge", "l") \
X(Ciugt, "a") \ X(Ciugt, "a", "be") \
X(Ciuge, "ae") \ X(Ciuge, "ae", "b") \
X(Cieq, "z") \ X(Cieq, "z", "nz") \
X(Cine, "nz") \ X(Cine, "nz", "z") \
X(NCmpI+Cfle, "be") \ X(NCmpI+Cfle, "?" , "?") \
X(NCmpI+Cflt, "b") \ X(NCmpI+Cflt, "?", "?") \
X(NCmpI+Cfgt, "a") \ X(NCmpI+Cfgt, "a", "be") \
X(NCmpI+Cfge, "ae") \ X(NCmpI+Cfge, "ae", "b") \
X(NCmpI+Cfeq, "z") \ X(NCmpI+Cfo, "np", "p") \
X(NCmpI+Cfne, "nz") \ X(NCmpI+Cfuo, "p", "np")
X(NCmpI+Cfo, "np") \
X(NCmpI+Cfuo, "p")
enum { enum {
SLong = 0, SLong = 0,
@ -124,13 +122,25 @@ static struct {
{ Oxcmp, Kd, "ucomisd %D0, %D1" }, { Oxcmp, Kd, "ucomisd %D0, %D1" },
{ Oxcmp, Ki, "cmp%k %0, %1" }, { Oxcmp, Ki, "cmp%k %0, %1" },
{ Oxtest, Ki, "test%k %0, %1" }, { Oxtest, Ki, "test%k %0, %1" },
#define X(c, s) \ #define X(c, s, _) \
{ Oflag+c, Ki, "set" s " %B=\n\tmovzb%k %B=, %=" }, { Oflag+c, Ki, "set" s " %B=\n\tmovzb%k %B=, %=" },
CMP(X) CMP(X)
#undef X #undef X
{ Oflagfeq, Ki, "setz %B=\n\tmovzb%k %B=, %=" },
{ Oflagfne, Ki, "setnz %B=\n\tmovzb%k %B=, %=" },
{ NOp, 0, 0 } { NOp, 0, 0 }
}; };
static char cmov[][2][16] = {
#define X(c, s0, s1) \
[c] = { \
"cmov" s0 " %0, %=", \
"cmov" s1 " %1, %=", \
},
CMP(X)
#undef X
};
static char *rname[][4] = { static char *rname[][4] = {
[RAX] = {"rax", "eax", "ax", "al"}, [RAX] = {"rax", "eax", "ax", "al"},
[RBX] = {"rbx", "ebx", "bx", "bl"}, [RBX] = {"rbx", "ebx", "bx", "bl"},
@ -167,9 +177,12 @@ slot(Ref r, E *e)
} }
else if (e->fp == RSP) else if (e->fp == RSP)
return 4*s + e->nclob*8; return 4*s + e->nclob*8;
else if (e->fn->vararg) else if (e->fn->vararg) {
return -176 + -4 * (e->fn->slot - s); if (T.windows)
return -4 * (e->fn->slot - s);
else else
return -176 + -4 * (e->fn->slot - s);
} else
return -4 * (e->fn->slot - s); return -4 * (e->fn->slot - s);
} }
@ -183,12 +196,12 @@ emitcon(Con *con, E *e)
l = str(con->sym.id); l = str(con->sym.id);
p = l[0] == '"' ? "" : T.assym; p = l[0] == '"' ? "" : T.assym;
if (con->sym.type == SThr) { if (con->sym.type == SThr) {
if (T.apple) assert(!T.apple);
fprintf(e->f, "%s%s@TLVP", p, l);
else
fprintf(e->f, "%%fs:%s%s@tpoff", p, l); fprintf(e->f, "%%fs:%s%s@tpoff", p, l);
} else } else {
assert((con->sym.type & ~SExt) == SGlo);
fprintf(e->f, "%s%s", p, l); fprintf(e->f, "%s%s", p, l);
}
if (con->bits.i) if (con->bits.i)
fprintf(e->f, "%+"PRId64, con->bits.i); fprintf(e->f, "%+"PRId64, con->bits.i);
break; break;
@ -367,7 +380,7 @@ Next:
off = e->fn->con[ref.val]; off = e->fn->con[ref.val];
emitcon(&off, e); emitcon(&off, e);
if (off.type == CAddr) if (off.type == CAddr)
if (off.sym.type != SThr || T.apple) if (off.sym.type != SThr)
fprintf(e->f, "(%%rip)"); fprintf(e->f, "(%%rip)");
break; break;
case RTmp: case RTmp:
@ -384,9 +397,9 @@ Next:
goto Next; goto Next;
} }
static void *negmask[4] = { static bits negmask[4] = {
[Ks] = (uint32_t[4]){ 0x80000000 }, [Ks] = 0x80000000,
[Kd] = (uint64_t[2]){ 0x8000000000000000 }, [Kd] = 0x8000000000000000,
}; };
static void static void
@ -401,6 +414,8 @@ emitins(Ins i, E *e)
switch (i.op) { switch (i.op) {
default: default:
if (isxsel(i.op))
goto case_Oxsel;
Table: Table:
/* most instructions are just pulled out of /* most instructions are just pulled out of
* the table omap[], some special cases are * the table omap[], some special cases are
@ -517,14 +532,22 @@ emitins(Ins i, E *e)
emitf("mov%k %0, %=", &i, e); emitf("mov%k %0, %=", &i, e);
break; break;
case Oaddr: case Oaddr:
if (!T.apple if (rtype(i.arg[0]) != RCon)
&& rtype(i.arg[0]) == RCon goto Table;
&& e->fn->con[i.arg[0].val].sym.type == SThr) { con = &e->fn->con[i.arg[0].val];
assert(isreg(i.to) && con->type == CAddr);
sym = str(con->sym.id);
if (T.apple && (con->sym.type & SThr)) {
fprintf(e->f,
"\tmovq %s%s@tlvp(%%rip), %%%s\n",
sym[0] == '"' ? "" : T.assym, sym,
regtoa(i.to.val, SLong));
break;
}
switch (con->sym.type) {
case SThr:
/* derive the symbol address from the TCB /* derive the symbol address from the TCB
* address at offset 0 of %fs */ * address at offset 0 of %fs */
assert(isreg(i.to));
con = &e->fn->con[i.arg[0].val];
sym = str(con->sym.id);
emitf("movq %%fs:0, %L=", &i, e); emitf("movq %%fs:0, %L=", &i, e);
fprintf(e->f, "\tleaq %s%s@tpoff", fprintf(e->f, "\tleaq %s%s@tpoff",
sym[0] == '"' ? "" : T.assym, sym); sym[0] == '"' ? "" : T.assym, sym);
@ -535,15 +558,40 @@ emitins(Ins i, E *e)
regtoa(i.to.val, SLong), regtoa(i.to.val, SLong),
regtoa(i.to.val, SLong)); regtoa(i.to.val, SLong));
break; break;
} case SExtThr:
/* initial-exec TLS: load offset from
* GOT, add to thread-base register */
assert(!con->bits.i);
emitf("movq %%fs:0, %L=", &i, e);
fprintf(e->f,
"\taddq %s%s@gottpoff(%%rip), %%%s\n",
sym[0] == '"' ? "" : T.assym, sym,
regtoa(i.to.val, SLong));
break;
case SExt:
/* load address from the GOT */
assert(!con->bits.i);
fprintf(e->f,
"\tmovq %s%s@gotpcrel(%%rip), %%%s\n",
sym[0] == '"' ? "" : T.assym, sym,
regtoa(i.to.val, SLong));
break;
default:
goto Table; goto Table;
}
break;
case Ocall: case Ocall:
/* calls simply have a weird syntax in AT&T /* calls simply have a weird syntax in AT&T
* assembly... */ * assembly... */
switch (rtype(i.arg[0])) { switch (rtype(i.arg[0])) {
case RCon: case RCon:
con = &e->fn->con[i.arg[0].val];
fprintf(e->f, "\tcallq "); fprintf(e->f, "\tcallq ");
emitcon(&e->fn->con[i.arg[0].val], e); emitcon(con, e);
if (con->type == CAddr
&& (con->sym.type & SExt)
&& !T.apple)
fprintf(e->f, "@plt");
fprintf(e->f, "\n"); fprintf(e->f, "\n");
break; break;
case RTmp: case RTmp:
@ -576,18 +624,27 @@ emitins(Ins i, E *e)
case Odbgloc: case Odbgloc:
emitdbgloc(i.arg[0].val, i.arg[1].val, e->f); emitdbgloc(i.arg[0].val, i.arg[1].val, e->f);
break; break;
case_Oxsel:
if (req(i.to, i.arg[1]))
emitf(cmov[i.op-Oxsel][0], &i, e);
else {
if (!req(i.to, i.arg[0]))
emitf("mov %0, %=", &i, e);
emitf(cmov[i.op-Oxsel][1], &i, e);
}
break;
} }
} }
static void static void
framesz(E *e) sysv_framesz(E *e)
{ {
uint64_t i, o, f; uint64_t i, o, f;
/* specific to NAlign == 3 */ /* specific to NAlign == 3 */
o = 0; o = 0;
if (!e->fn->leaf) { if (!e->fn->leaf) {
for (i=0, o=0; i<NCLR; i++) for (i=0, o=0; i<NCLR_SYSV; i++)
o ^= e->fn->reg >> amd64_sysv_rclob[i]; o ^= e->fn->reg >> amd64_sysv_rclob[i];
o &= 1; o &= 1;
} }
@ -601,10 +658,10 @@ framesz(E *e)
} }
void void
amd64_emitfn(Fn *fn, FILE *f) amd64_sysv_emitfn(Fn *fn, FILE *f)
{ {
static char *ctoa[] = { static char *ctoa[][2] = {
#define X(c, s) [c] = s, #define X(c, s, n) [c] = {s, n},
CMP(X) CMP(X)
#undef X #undef X
}; };
@ -623,7 +680,7 @@ amd64_emitfn(Fn *fn, FILE *f)
fputs("\tpushq %rbp\n\tmovq %rsp, %rbp\n", f); fputs("\tpushq %rbp\n\tmovq %rsp, %rbp\n", f);
} else } else
e->fp = RSP; e->fp = RSP;
framesz(e); sysv_framesz(e);
if (e->fsz) if (e->fsz)
fprintf(f, "\tsubq $%"PRIu64", %%rsp\n", e->fsz); fprintf(f, "\tsubq $%"PRIu64", %%rsp\n", e->fsz);
if (fn->vararg) { if (fn->vararg) {
@ -633,7 +690,7 @@ amd64_emitfn(Fn *fn, FILE *f)
for (n=0; n<8; ++n, o+=16) for (n=0; n<8; ++n, o+=16)
fprintf(f, "\tmovaps %%xmm%d, %d(%%rbp)\n", n, o); fprintf(f, "\tmovaps %%xmm%d, %d(%%rbp)\n", n, o);
} }
for (r=amd64_sysv_rclob; r<&amd64_sysv_rclob[NCLR]; r++) for (r=amd64_sysv_rclob; r<&amd64_sysv_rclob[NCLR_SYSV]; r++)
if (fn->reg & BIT(*r)) { if (fn->reg & BIT(*r)) {
itmp.arg[0] = TMP(*r); itmp.arg[0] = TMP(*r);
emitf("pushq %L0", &itmp, e); emitf("pushq %L0", &itmp, e);
@ -662,7 +719,7 @@ amd64_emitfn(Fn *fn, FILE *f)
"\tmovq %%rbp, %%rsp\n" "\tmovq %%rbp, %%rsp\n"
"\tsubq $%"PRIu64", %%rsp\n", "\tsubq $%"PRIu64", %%rsp\n",
e->fsz + e->nclob * 8); e->fsz + e->nclob * 8);
for (r=&amd64_sysv_rclob[NCLR]; r>amd64_sysv_rclob;) for (r=&amd64_sysv_rclob[NCLR_SYSV]; r>amd64_sysv_rclob;)
if (fn->reg & BIT(*--r)) { if (fn->reg & BIT(*--r)) {
itmp.arg[0] = TMP(*r); itmp.arg[0] = TMP(*r);
emitf("popq %L0", &itmp, e); emitf("popq %L0", &itmp, e);
@ -690,9 +747,10 @@ amd64_emitfn(Fn *fn, FILE *f)
s = b->s1; s = b->s1;
b->s1 = b->s2; b->s1 = b->s2;
b->s2 = s; b->s2 = s;
n = 0;
} else } else
c = cmpneg(c); n = 1;
fprintf(f, "\tj%s %sbb%d\n", ctoa[c], fprintf(f, "\tj%s %sbb%d\n", ctoa[c][n],
T.asloc, id0+b->s2->id); T.asloc, id0+b->s2->id);
goto Jmp; goto Jmp;
} }
@ -703,3 +761,119 @@ amd64_emitfn(Fn *fn, FILE *f)
if (!T.apple) if (!T.apple)
elf_emitfnfin(fn->name, f); elf_emitfnfin(fn->name, f);
} }
static void
winabi_framesz(E *e)
{
uint64_t i, o, f;
/* specific to NAlign == 3 */
o = 0;
if (!e->fn->leaf) {
for (i=0, o=0; i<NCLR_WIN; i++)
o ^= e->fn->reg >> amd64_winabi_rclob[i];
o &= 1;
}
f = e->fn->slot;
f = (f + 3) & -4;
if (f > 0
&& e->fp == RSP
&& e->fn->salign == 4)
f += 2;
e->fsz = 4*f + 8*o;
}
void
amd64_winabi_emitfn(Fn *fn, FILE *f)
{
static char *ctoa[][2] = {
#define X(c, s, n) [c] = {s, n},
CMP(X)
#undef X
};
static int id0;
Blk *b, *s;
Ins *i, itmp;
int *r, c, n, lbl;
E *e;
e = &(E){.f = f, .fn = fn};
emitfnlnk(fn->name, &fn->lnk, f);
fputs("\tendbr64\n", f);
if (fn->vararg) {
fprintf(f, "\tmovq %%rcx, 0x8(%%rsp)\n");
fprintf(f, "\tmovq %%rdx, 0x10(%%rsp)\n");
fprintf(f, "\tmovq %%r8, 0x18(%%rsp)\n");
fprintf(f, "\tmovq %%r9, 0x20(%%rsp)\n");
}
if (!fn->leaf || fn->vararg || fn->dynalloc) {
e->fp = RBP;
fputs("\tpushq %rbp\n\tmovq %rsp, %rbp\n", f);
} else
e->fp = RSP;
winabi_framesz(e);
if (e->fsz)
fprintf(f, "\tsubq $%"PRIu64", %%rsp\n", e->fsz);
for (r=amd64_winabi_rclob; r<&amd64_winabi_rclob[NCLR_WIN]; r++)
if (fn->reg & BIT(*r)) {
itmp.arg[0] = TMP(*r);
emitf("pushq %L0", &itmp, e);
e->nclob++;
}
for (lbl=0, b=fn->start; b; b=b->link) {
if (lbl || b->npred > 1)
fprintf(f, "%sbb%d:\n", T.asloc, id0+b->id);
for (i=b->ins; i!=&b->ins[b->nins]; i++)
emitins(*i, e);
lbl = 1;
switch (b->jmp.type) {
case Jhlt:
fprintf(f, "\tud2\n");
break;
case Jret0:
if (fn->dynalloc)
fprintf(f,
"\tmovq %%rbp, %%rsp\n"
"\tsubq $%"PRIu64", %%rsp\n",
e->fsz + e->nclob * 8);
for (r=&amd64_winabi_rclob[NCLR_WIN]; r>amd64_winabi_rclob;)
if (fn->reg & BIT(*--r)) {
itmp.arg[0] = TMP(*r);
emitf("popq %L0", &itmp, e);
}
if (e->fp == RBP)
fputs("\tleave\n", f);
else if (e->fsz)
fprintf(f,
"\taddq $%"PRIu64", %%rsp\n",
e->fsz);
fputs("\tret\n", f);
break;
case Jjmp:
Jmp:
if (b->s1 != b->link)
fprintf(f, "\tjmp %sbb%d\n",
T.asloc, id0+b->s1->id);
else
lbl = 0;
break;
default:
c = b->jmp.type - Jjf;
if (0 <= c && c <= NCmp) {
if (b->link == b->s2 || c >= NCmpI) {
s = b->s1;
b->s1 = b->s2;
b->s2 = s;
n = 0;
} else
n = 1;
fprintf(f, "\tj%s %sbb%d\n", ctoa[c][n],
T.asloc, id0+b->s2->id);
goto Jmp;
}
die("unhandled jump %d", b->jmp.type);
}
}
id0 += fn->nblk;
}

View File

@ -87,7 +87,7 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
vgrow(&fn->mem, ++fn->nmem); vgrow(&fn->mem, ++fn->nmem);
memset(&a, 0, sizeof a); memset(&a, 0, sizeof a);
a.offset.type = CAddr; a.offset.type = CAddr;
n = stashbits(&fn->con[r0.val].bits, KWIDE(k) ? 8 : 4); n = stashbits(fn->con[r0.val].bits.i, KWIDE(k) ? 8 : 4);
/* quote the name so that we do not /* quote the name so that we do not
* add symbol prefixes on the apple * add symbol prefixes on the apple
* target variant * target variant
@ -96,6 +96,14 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
a.offset.sym.id = intern(buf); a.offset.sym.id = intern(buf);
fn->mem[fn->nmem-1] = a; fn->mem[fn->nmem-1] = a;
} }
else if (op == Ocall && r == &i->arg[0]
&& rtype(r0) == RCon && fn->con[r0.val].type != CAddr) {
/* use a temporary register so that we
* produce an indirect call
*/
r1 = newtmp("isel", Kl, fn);
emit(Ocopy, Kl, r1, r0, R);
}
else if (op != Ocopy && k == Kl && noimm(r0, fn)) { else if (op != Ocopy && k == Kl && noimm(r0, fn)) {
/* load constants that do not fit in /* load constants that do not fit in
* a 32bit signed integer into a * a 32bit signed integer into a
@ -112,8 +120,9 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
r1 = newtmp("isel", Kl, fn); r1 = newtmp("isel", Kl, fn);
emit(Oaddr, Kl, r1, SLOT(s), R); emit(Oaddr, Kl, r1, SLOT(s), R);
} }
else if (T.apple && hascon(r0, &c, fn) else if (op != Ocall && hascon(r0, &c, fn)
&& c->type == CAddr && c->sym.type == SThr) { && c->type == CAddr && ((c->sym.type & SExt)
|| (T.apple && c->sym.type == SThr))) {
r1 = newtmp("isel", Kl, fn); r1 = newtmp("isel", Kl, fn);
if (c->bits.i) { if (c->bits.i) {
r2 = newtmp("isel", Kl, fn); r2 = newtmp("isel", Kl, fn);
@ -123,16 +132,18 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
emit(Oadd, Kl, r1, r2, r3); emit(Oadd, Kl, r1, r2, r3);
} else } else
r2 = r1; r2 = r1;
if (T.apple && (c->sym.type & SThr)) {
emit(Ocopy, Kl, r2, TMP(RAX), R); emit(Ocopy, Kl, r2, TMP(RAX), R);
r2 = newtmp("isel", Kl, fn); r2 = newtmp("isel", Kl, fn);
r3 = newtmp("isel", Kl, fn); r3 = newtmp("isel", Kl, fn);
emit(Ocall, 0, R, r3, CALL(17)); emit(Ocall, 0, R, r3, CALL(17));
emit(Ocopy, Kl, TMP(RDI), r2, R); emit(Ocopy, Kl, TMP(RDI), r2, R);
emit(Oload, Kl, r3, r2, R); emit(Oload, Kl, r3, r2, R);
}
cc = *c; cc = *c;
cc.bits.i = 0; cc.bits.i = 0;
r3 = newcon(&cc, fn); r3 = newcon(&cc, fn);
emit(Oload, Kl, r2, r3, R); emit(Oaddr, Kl, r2, r3, R);
if (rtype(r0) == RMem) { if (rtype(r0) == RMem) {
m = &fn->mem[r0.val]; m = &fn->mem[r0.val];
m->offset.type = CUndef; m->offset.type = CUndef;
@ -143,9 +154,8 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
else if (!(isstore(op) && r == &i->arg[1]) else if (!(isstore(op) && r == &i->arg[1])
&& !isload(op) && op != Ocall && rtype(r0) == RCon && !isload(op) && op != Ocall && rtype(r0) == RCon
&& fn->con[r0.val].type == CAddr) { && fn->con[r0.val].type == CAddr) {
/* apple as does not support 32-bit /* turn address operands into
* absolute addressing, use a rip- * lea/mov instructions
* relative leaq instead
*/ */
r1 = newtmp("isel", Kl, fn); r1 = newtmp("isel", Kl, fn);
emit(Oaddr, Kl, r1, r0, R); emit(Oaddr, Kl, r1, r0, R);
@ -163,6 +173,10 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
m->base = r0; m->base = r0;
} }
} }
else if (isxsel(op) && rtype(*r) == RCon) {
r1 = newtmp("isel", i->cls, fn);
emit(Ocopy, i->cls, r1, *r, R);
}
*r = r1; *r = r1;
} }
@ -425,7 +439,8 @@ sel(Ins i, Num *tn, Fn *fn)
case Oexts: case Oexts:
case Otruncd: case Otruncd:
case Ocast: case Ocast:
case_OExt: case_Oxsel:
case_Oext:
Emit: Emit:
emiti(i); emiti(i);
i1 = curi; /* fixarg() can change curi */ i1 = curi; /* fixarg() can change curi */
@ -439,7 +454,9 @@ Emit:
break; break;
default: default:
if (isext(i.op)) if (isext(i.op))
goto case_OExt; goto case_Oext;
if (isxsel(i.op))
goto case_Oxsel;
if (isload(i.op)) if (isload(i.op))
goto case_Oload; goto case_Oload;
if (iscmp(i.op, &kc, &x)) { if (iscmp(i.op, &kc, &x)) {
@ -493,6 +510,88 @@ flagi(Ins *i0, Ins *i)
return 0; return 0;
} }
static Ins*
selsel(Fn *fn, Blk *b, Ins *i, Num *tn)
{
Ref r, cr[2];
int c, k, swap, gencmp, gencpy;
Ins *isel0, *isel1, *fi;
Tmp *t;
assert(i->op == Osel1);
for (isel0=i; b->ins<isel0; isel0--) {
if (isel0->op == Osel0)
break;
assert(isel0->op == Osel1);
}
assert(isel0->op == Osel0);
r = isel0->arg[0];
assert(rtype(r) == RTmp);
t = &fn->tmp[r.val];
fi = flagi(b->ins, isel0);
cr[0] = cr[1] = R;
gencmp = gencpy = swap = 0;
k = Kw;
c = Cine;
if (!fi || !req(fi->to, r)) {
gencmp = 1;
cr[0] = r;
cr[1] = CON_Z;
}
else if (iscmp(fi->op, &k, &c)) {
if (c == NCmpI+Cfeq
|| c == NCmpI+Cfne) {
/* these are selected as 'and'
* or 'or', so we check their
* result with Cine
*/
c = Cine;
goto Other;
}
swap = cmpswap(fi->arg, c);
if (swap)
c = cmpop(c);
if (t->nuse == 1) {
gencmp = 1;
cr[0] = fi->arg[0];
cr[1] = fi->arg[1];
*fi = (Ins){.op = Onop};
}
}
else if (fi->op == Oand && t->nuse == 1
&& (rtype(fi->arg[0]) == RTmp ||
rtype(fi->arg[1]) == RTmp)) {
fi->op = Oxtest;
fi->to = R;
if (rtype(fi->arg[1]) == RCon) {
r = fi->arg[1];
fi->arg[1] = fi->arg[0];
fi->arg[0] = r;
}
}
else {
Other:
/* since flags are not tracked in liveness,
* the result of the flag-setting instruction
* has to be marked as live
*/
if (t->nuse == 1)
gencpy = 1;
}
/* generate conditional moves */
for (isel1=i; isel0<isel1; --isel1) {
isel1->op = Oxsel+c;
sel(*isel1, tn, fn);
}
assert(!gencmp || !gencpy);
if (gencmp)
selcmp(cr, k, swap, fn);
if (gencpy)
emit(Ocopy, Kw, R, r, R);
*isel0 = (Ins){.op = Onop};
return isel0;
}
static void static void
seljmp(Blk *b, Fn *fn) seljmp(Blk *b, Fn *fn)
{ {
@ -522,7 +621,7 @@ seljmp(Blk *b, Fn *fn)
b->jmp.type = Jjf + Cine; b->jmp.type = Jjf + Cine;
} }
else if (iscmp(fi->op, &k, &c) else if (iscmp(fi->op, &k, &c)
&& c != NCmpI+Cfeq /* see sel() */ && c != NCmpI+Cfeq /* see sel(), selsel() */
&& c != NCmpI+Cfne) { && c != NCmpI+Cfne) {
swap = cmpswap(fi->arg, c); swap = cmpswap(fi->arg, c);
if (swap) if (swap)
@ -826,8 +925,14 @@ amd64_isel(Fn *fn)
memset(num, 0, n * sizeof num[0]); memset(num, 0, n * sizeof num[0]);
anumber(num, b, fn->con); anumber(num, b, fn->con);
seljmp(b, fn); seljmp(b, fn);
for (i=&b->ins[b->nins]; i!=b->ins;) for (i=&b->ins[b->nins]; i!=b->ins;) {
sel(*--i, num, fn); --i;
assert(i->op != Osel0);
if (i->op == Osel1)
i = selsel(fn, b, i, num);
else
sel(*i, num, fn);
}
idup(b, curi, &insb[NIns]-curi); idup(b, curi, &insb[NIns]-curi);
} }
free(num); free(num);

View File

@ -228,8 +228,8 @@ int amd64_sysv_rsave[] = {
int amd64_sysv_rclob[] = {RBX, R12, R13, R14, R15, -1}; int amd64_sysv_rclob[] = {RBX, R12, R13, R14, R15, -1};
MAKESURE(sysv_arrays_ok, MAKESURE(sysv_arrays_ok,
sizeof amd64_sysv_rsave == (NGPS+NFPS+1) * sizeof(int) && sizeof amd64_sysv_rsave == (NGPS_SYSV+NFPS+1) * sizeof(int) &&
sizeof amd64_sysv_rclob == (NCLR+1) * sizeof(int) sizeof amd64_sysv_rclob == (NCLR_SYSV+1) * sizeof(int)
); );
/* layout of call's second argument (RCall) /* layout of call's second argument (RCall)

View File

@ -19,20 +19,21 @@ amd64_memargs(int op)
.nfpr = NFPR, \ .nfpr = NFPR, \
.rglob = BIT(RBP) | BIT(RSP), \ .rglob = BIT(RBP) | BIT(RSP), \
.nrglob = 2, \ .nrglob = 2, \
.rsave = amd64_sysv_rsave, \
.nrsave = {NGPS, NFPS}, \
.retregs = amd64_sysv_retregs, \
.argregs = amd64_sysv_argregs, \
.memargs = amd64_memargs, \ .memargs = amd64_memargs, \
.abi0 = elimsb, \ .abi0 = elimsb, \
.abi1 = amd64_sysv_abi, \
.isel = amd64_isel, \ .isel = amd64_isel, \
.emitfn = amd64_emitfn, \ .cansel = 1,
Target T_amd64_sysv = { Target T_amd64_sysv = {
.name = "amd64_sysv", .name = "amd64_sysv",
.emitfin = elf_emitfin, .emitfin = elf_emitfin,
.asloc = ".L", .asloc = ".L",
.abi1 = amd64_sysv_abi,
.rsave = amd64_sysv_rsave,
.nrsave = {NGPS_SYSV, NFPS},
.retregs = amd64_sysv_retregs,
.argregs = amd64_sysv_argregs,
.emitfn = amd64_sysv_emitfn,
AMD64_COMMON AMD64_COMMON
}; };
@ -42,5 +43,25 @@ Target T_amd64_apple = {
.emitfin = macho_emitfin, .emitfin = macho_emitfin,
.asloc = "L", .asloc = "L",
.assym = "_", .assym = "_",
.abi1 = amd64_sysv_abi,
.rsave = amd64_sysv_rsave,
.nrsave = {NGPS_SYSV, NFPS},
.retregs = amd64_sysv_retregs,
.argregs = amd64_sysv_argregs,
.emitfn = amd64_sysv_emitfn,
AMD64_COMMON
};
Target T_amd64_win = {
.name = "amd64_win",
.windows = 1,
.emitfin = pe_emitfin,
.asloc = "L",
.abi1 = amd64_winabi_abi,
.rsave = amd64_winabi_rsave,
.nrsave = {NGPS_WIN, NFPS},
.retregs = amd64_winabi_retregs,
.argregs = amd64_winabi_argregs,
.emitfn = amd64_winabi_emitfn,
AMD64_COMMON AMD64_COMMON
}; };

763
amd64/winabi.c Executable file
View File

@ -0,0 +1,763 @@
#include "all.h"
#include <stdbool.h>
typedef enum ArgPassStyle {
APS_Invalid = 0,
APS_Register,
APS_InlineOnStack,
APS_CopyAndPointerInRegister,
APS_CopyAndPointerOnStack,
APS_VarargsTag,
APS_EnvTag,
} ArgPassStyle;
typedef struct ArgClass {
Typ* type;
ArgPassStyle style;
int align;
uint size;
int cls;
Ref ref;
} ArgClass;
typedef struct ExtraAlloc ExtraAlloc;
struct ExtraAlloc {
Ins instr;
ExtraAlloc* link;
};
#define ALIGN_DOWN(n, a) ((n) & ~((a)-1))
#define ALIGN_UP(n, a) ALIGN_DOWN((n) + (a)-1, (a))
// Number of stack bytes required be reserved for the callee.
#define SHADOW_SPACE_SIZE 32
int amd64_winabi_rsave[] = {RCX, RDX, R8, R9, R10, R11, RAX, XMM0,
XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7, XMM8,
XMM9, XMM10, XMM11, XMM12, XMM13, XMM14, -1};
int amd64_winabi_rclob[] = {RBX, R12, R13, R14, R15, RSI, RDI, -1};
MAKESURE(winabi_arrays_ok,
sizeof amd64_winabi_rsave == (NGPS_WIN + NFPS + 1) * sizeof(int) &&
sizeof amd64_winabi_rclob == (NCLR_WIN + 1) * sizeof(int));
// layout of call's second argument (RCall)
//
// bit 0: rax returned
// bit 1: xmm0 returned
// bits 23: 0
// bits 4567: rcx, rdx, r8, r9 passed
// bits 89ab: xmm0,1,2,3 passed
// bit c: env call (rax passed)
// bits d..1f: 0
bits amd64_winabi_retregs(Ref r, int p[2]) {
assert(rtype(r) == RCall);
bits b = 0;
int num_int_returns = r.val & 1;
int num_float_returns = r.val & 2;
if (num_int_returns == 1) {
b |= BIT(RAX);
} else {
b |= BIT(XMM0);
}
if (p) {
p[0] = num_int_returns;
p[1] = num_float_returns;
}
return b;
}
static uint popcnt(bits b) {
b = (b & 0x5555555555555555) + ((b >> 1) & 0x5555555555555555);
b = (b & 0x3333333333333333) + ((b >> 2) & 0x3333333333333333);
b = (b & 0x0f0f0f0f0f0f0f0f) + ((b >> 4) & 0x0f0f0f0f0f0f0f0f);
b += (b >> 8);
b += (b >> 16);
b += (b >> 32);
return b & 0xff;
}
bits amd64_winabi_argregs(Ref r, int p[2]) {
assert(rtype(r) == RCall);
// On SysV, these are counts. Here, a count isn't sufficient, we actually need
// to know which ones are in use because they're not necessarily contiguous.
int int_passed = (r.val >> 4) & 15;
int float_passed = (r.val >> 8) & 15;
bool env_param = (r.val >> 12) & 1;
bits b = 0;
b |= (int_passed & 1) ? BIT(RCX) : 0;
b |= (int_passed & 2) ? BIT(RDX) : 0;
b |= (int_passed & 4) ? BIT(R8) : 0;
b |= (int_passed & 8) ? BIT(R9) : 0;
b |= (float_passed & 1) ? BIT(XMM0) : 0;
b |= (float_passed & 2) ? BIT(XMM1) : 0;
b |= (float_passed & 4) ? BIT(XMM2) : 0;
b |= (float_passed & 8) ? BIT(XMM3) : 0;
b |= env_param ? BIT(RAX) : 0;
if (p) {
// TODO: The only place this is used is live.c. I'm not sure what should be
// returned here wrt to using the same counter for int/float regs on win.
// For now, try the number of registers in use even though they're not
// contiguous.
p[0] = popcnt(int_passed);
p[1] = popcnt(float_passed);
}
return b;
}
typedef struct RegisterUsage {
// Counter for both int/float as they're counted together. Only if the bool's
// set in regs_passed is the given register *actually* needed for a value
// (i.e. needs to be saved, etc.).
int num_regs_passed;
// Indexed first by 0=int, 1=float, use KBASE(cls).
// Indexed second by register index in calling convention, so for integer,
// 0=RCX, 1=RDX, 2=R8, 3=R9, and for float XMM0, XMM1, XMM2, XMM3.
bool regs_passed[2][4];
bool rax_returned;
bool xmm0_returned;
// This is also used as where the va_start will start for varargs functions
// (there's no 'Oparv', so we need to keep track of a count here.)
int num_named_args_passed;
// This is set when classifying the arguments for a call (but not when
// classifying the parameters of a function definition).
bool is_varargs_call;
bool has_env;
} RegisterUsage;
static int register_usage_to_call_arg_value(RegisterUsage reg_usage) {
return (reg_usage.rax_returned << 0) | //
(reg_usage.xmm0_returned << 1) | //
(reg_usage.regs_passed[0][0] << 4) | //
(reg_usage.regs_passed[0][1] << 5) | //
(reg_usage.regs_passed[0][2] << 6) | //
(reg_usage.regs_passed[0][3] << 7) | //
(reg_usage.regs_passed[1][0] << 8) | //
(reg_usage.regs_passed[1][1] << 9) | //
(reg_usage.regs_passed[1][2] << 10) | //
(reg_usage.regs_passed[1][3] << 11) | //
(reg_usage.has_env << 12);
}
// Assigns the argument to a register if there's any left according to the
// calling convention, and updates the regs_passed bools. Otherwise marks the
// value as needing stack space to be passed.
static void assign_register_or_stack(RegisterUsage* reg_usage,
ArgClass* arg,
bool is_float,
bool by_copy) {
if (reg_usage->num_regs_passed == 4) {
arg->style = by_copy ? APS_CopyAndPointerOnStack : APS_InlineOnStack;
} else {
reg_usage->regs_passed[is_float][reg_usage->num_regs_passed] = true;
++reg_usage->num_regs_passed;
arg->style = by_copy ? APS_CopyAndPointerInRegister : APS_Register;
}
++reg_usage->num_named_args_passed;
}
static bool type_is_by_copy(Typ* type) {
// Note that only these sizes are passed by register, even though e.g. a
// 5 byte struct would "fit", it still is passed by copy-and-pointer.
return type->isdark || (type->size != 1 && type->size != 2 &&
type->size != 4 && type->size != 8);
}
// This function is used for both arguments and parameters.
// begin_instr should either point at the first Oarg or Opar, and end_instr
// should point past the last one (so to the Ocall for arguments, or to the
// first 'real' instruction of the function for parameters).
static void classify_arguments(RegisterUsage* reg_usage,
Ins* begin_instr,
Ins* end_instr,
ArgClass* arg_classes,
Ref* env) {
ArgClass* arg = arg_classes;
// For each argument, determine how it will be passed (int, float, stack)
// and update the `reg_usage` counts. Additionally, fill out arg_classes for
// each argument.
for (Ins* instr = begin_instr; instr < end_instr; ++instr, ++arg) {
switch (instr->op) {
case Oarg:
case Opar:
assign_register_or_stack(reg_usage, arg, KBASE(instr->cls),
/*by_copy=*/false);
arg->cls = instr->cls;
arg->align = 3;
arg->size = 8;
break;
case Oargc:
case Oparc: {
int typ_index = instr->arg[0].val;
Typ* type = &typ[typ_index];
bool by_copy = type_is_by_copy(type);
assign_register_or_stack(reg_usage, arg, /*is_float=*/false, by_copy);
arg->cls = Kl;
if (!by_copy && type->size <= 4) {
arg->cls = Kw;
}
arg->align = 3;
arg->size = type->size;
break;
}
case Oarge:
*env = instr->arg[0];
arg->style = APS_EnvTag;
reg_usage->has_env = true;
break;
case Opare:
*env = instr->to;
arg->style = APS_EnvTag;
reg_usage->has_env = true;
break;
case Oargv:
reg_usage->is_varargs_call = true;
arg->style = APS_VarargsTag;
break;
}
}
if (reg_usage->has_env && reg_usage->is_varargs_call) {
die("can't use env with varargs");
}
// During a varargs call, float arguments have to be duplicated to their
// associated integer register, so mark them as in-use too.
if (reg_usage->is_varargs_call) {
for (int i = 0; i < 4; ++i) {
if (reg_usage->regs_passed[/*float*/ 1][i]) {
reg_usage->regs_passed[/*int*/ 0][i] = true;
}
}
}
}
static bool is_integer_type(int ty) {
assert(ty >= 0 && ty < 4 && "expecting Kw Kl Ks Kd");
return KBASE(ty) == 0;
}
static Ref register_for_arg(int cls, int counter) {
assert(counter < 4);
if (is_integer_type(cls)) {
return TMP(amd64_winabi_rsave[counter]);
} else {
return TMP(XMM0 + counter);
}
}
static Ins* lower_call(Fn* func,
Blk* block,
Ins* call_instr,
ExtraAlloc** pextra_alloc) {
// Call arguments are instructions. Walk through them to find the end of the
// call+args that we need to process (and return the instruction past the body
// of the instruction for continuing processing).
Ins* instr_past_args = call_instr - 1;
for (; instr_past_args >= block->ins; --instr_past_args) {
if (!isarg(instr_past_args->op)) {
break;
}
}
Ins* earliest_arg_instr = instr_past_args + 1;
// Don't need an ArgClass for the call itself, so one less than the total
// number of instructions we're dealing with.
uint num_args = call_instr - earliest_arg_instr;
ArgClass* arg_classes = alloc(num_args * sizeof(ArgClass));
RegisterUsage reg_usage = {0};
ArgClass ret_arg_class = {0};
// Ocall's two arguments are the the function to be called in 0, and, if the
// the function returns a non-basic type, then arg[1] is a reference to the
// type of the return. req checks if Refs are equal; `R` is 0.
bool il_has_struct_return = !req(call_instr->arg[1], R);
bool is_struct_return = false;
if (il_has_struct_return) {
Typ* ret_type = &typ[call_instr->arg[1].val];
is_struct_return = type_is_by_copy(ret_type);
if (is_struct_return) {
assign_register_or_stack(&reg_usage, &ret_arg_class, /*is_float=*/false,
/*by_copy=*/true);
}
ret_arg_class.size = ret_type->size;
}
Ref env = R;
classify_arguments(&reg_usage, earliest_arg_instr, call_instr, arg_classes,
&env);
// We now know which arguments are on the stack and which are in registers, so
// we can allocate the correct amount of space to stash the stack-located ones
// into.
uint stack_usage = 0;
for (uint i = 0; i < num_args; ++i) {
ArgClass* arg = &arg_classes[i];
// stack_usage only accounts for pushes that are for values that don't have
// enough registers. Large struct copies are alloca'd separately, and then
// only have (potentially) 8 bytes to add to stack_usage here.
if (arg->style == APS_InlineOnStack) {
if (arg->align > 4) {
err("win abi cannot pass alignments > 16");
}
stack_usage += arg->size;
} else if (arg->style == APS_CopyAndPointerOnStack) {
stack_usage += 8;
}
}
stack_usage = ALIGN_UP(stack_usage, 16);
// Note that here we're logically 'after' the call (due to emitting
// instructions in reverse order), so we're doing a negative stack
// allocation to clean up after the call.
Ref stack_size_ref =
getcon(-(int64_t)(stack_usage + SHADOW_SPACE_SIZE), func);
emit(Osalloc, Kl, R, stack_size_ref, R);
ExtraAlloc* return_pad = NULL;
if (is_struct_return) {
return_pad = alloc(sizeof(ExtraAlloc));
Ref ret_pad_ref = newtmp("abi.ret_pad", Kl, func);
return_pad->instr =
(Ins){Oalloc8, Kl, ret_pad_ref, {getcon(ret_arg_class.size, func)}};
return_pad->link = (*pextra_alloc);
*pextra_alloc = return_pad;
reg_usage.rax_returned = true;
emit(Ocopy, call_instr->cls, call_instr->to, TMP(RAX), R);
} else {
if (il_has_struct_return) {
// In the case that at the IL level, a struct return was specified, but as
// far as the calling convention is concerned it's not actually by
// pointer, we need to store the return value into an alloca because
// subsequent IL will still be treating the function return as a pointer.
ExtraAlloc* return_copy = alloc(sizeof(ExtraAlloc));
return_copy->instr =
(Ins){Oalloc8, Kl, call_instr->to, {getcon(8, func)}};
return_copy->link = (*pextra_alloc);
*pextra_alloc = return_copy;
Ref copy = newtmp("abi.copy", Kl, func);
emit(Ostorel, 0, R, copy, call_instr->to);
emit(Ocopy, Kl, copy, TMP(RAX), R);
reg_usage.rax_returned = true;
} else if (is_integer_type(call_instr->cls)) {
// Only a basic type returned from the call, integer.
emit(Ocopy, call_instr->cls, call_instr->to, TMP(RAX), R);
reg_usage.rax_returned = true;
} else {
// Basic type, floating point.
emit(Ocopy, call_instr->cls, call_instr->to, TMP(XMM0), R);
reg_usage.xmm0_returned = true;
}
}
// Emit the actual call instruction. There's no 'to' value by this point
// because we've lowered it into register manipulation (that's the `R`),
// arg[0] of the call is the function, and arg[1] is register usage is
// documented as above (copied from SysV).
emit(Ocall, call_instr->cls, R, call_instr->arg[0],
CALL(register_usage_to_call_arg_value(reg_usage)));
if (!req(R, env)) {
// If there's an env arg to be passed, it gets stashed in RAX.
emit(Ocopy, Kl, TMP(RAX), env, R);
}
if (reg_usage.is_varargs_call) {
// Any float arguments need to be duplicated to integer registers. This is
// required by the calling convention so that dumping to shadow space can be
// done without a prototype and for varargs.
#define DUP_IF_USED(index, floatreg, intreg) \
if (reg_usage.regs_passed[/*float*/ 1][index]) { \
emit(Ocast, Kl, TMP(intreg), TMP(floatreg), R); \
}
DUP_IF_USED(0, XMM0, RCX);
DUP_IF_USED(1, XMM1, RDX);
DUP_IF_USED(2, XMM2, R8);
DUP_IF_USED(3, XMM3, R9);
#undef DUP_IF_USED
}
int reg_counter = 0;
if (is_struct_return) {
Ref first_reg = register_for_arg(Kl, reg_counter++);
emit(Ocopy, Kl, first_reg, return_pad->instr.to, R);
}
// This is where we actually do the load of values into registers or into
// stack slots.
Ref arg_stack_slots = newtmp("abi.args", Kl, func);
uint slot_offset = SHADOW_SPACE_SIZE;
ArgClass* arg = arg_classes;
for (Ins* instr = earliest_arg_instr; instr != call_instr; ++instr, ++arg) {
switch (arg->style) {
case APS_Register: {
Ref into = register_for_arg(arg->cls, reg_counter++);
if (instr->op == Oargc) {
// If this is a small struct being passed by value. The value in the
// instruction in this case is a pointer, but it needs to be loaded
// into the register.
emit(Oload, arg->cls, into, instr->arg[1], R);
} else {
// Otherwise, a normal value passed in a register.
emit(Ocopy, instr->cls, into, instr->arg[0], R);
}
break;
}
case APS_InlineOnStack: {
Ref slot = newtmp("abi.off", Kl, func);
if (instr->op == Oargc) {
// This is a small struct, so it's not passed by copy, but the
// instruction is a pointer. So we need to copy it into the stack
// slot. (And, remember that these are emitted backwards, so store,
// then load.)
Ref smalltmp = newtmp("abi.smalltmp", arg->cls, func);
emit(Ostorel, 0, R, smalltmp, slot);
emit(Oload, arg->cls, smalltmp, instr->arg[1], R);
} else {
// Stash the value into the stack slot.
emit(Ostorel, 0, R, instr->arg[0], slot);
}
emit(Oadd, Kl, slot, arg_stack_slots, getcon(slot_offset, func));
slot_offset += arg->size;
break;
}
case APS_CopyAndPointerInRegister:
case APS_CopyAndPointerOnStack: {
// Alloca a space to copy into, and blit the value from the instr to the
// copied location.
ExtraAlloc* arg_copy = alloc(sizeof(ExtraAlloc));
Ref copy_ref = newtmp("abi.copy", Kl, func);
arg_copy->instr =
(Ins){Oalloc8, Kl, copy_ref, {getcon(arg->size, func)}};
arg_copy->link = (*pextra_alloc);
*pextra_alloc = arg_copy;
emit(Oblit1, 0, R, INT(arg->size), R);
emit(Oblit0, 0, R, instr->arg[1], copy_ref);
// Now load the pointer into the correct register or stack slot.
if (arg->style == APS_CopyAndPointerInRegister) {
Ref into = register_for_arg(arg->cls, reg_counter++);
emit(Ocopy, Kl, into, copy_ref, R);
} else {
assert(arg->style == APS_CopyAndPointerOnStack);
Ref slot = newtmp("abi.off", Kl, func);
emit(Ostorel, 0, R, copy_ref, slot);
emit(Oadd, Kl, slot, arg_stack_slots, getcon(slot_offset, func));
slot_offset += 8;
}
break;
}
case APS_EnvTag:
case APS_VarargsTag:
// Nothing to do here, see right before the call for reg dupe.
break;
case APS_Invalid:
die("unreachable");
}
}
if (stack_usage) {
// The last (first in call order) thing we do is allocate the the stack
// space we're going to fill with temporaries.
emit(Osalloc, Kl, arg_stack_slots,
getcon(stack_usage + SHADOW_SPACE_SIZE, func), R);
} else {
// When there's no usage for temporaries, we can add this into the other
// alloca, but otherwise emit it separately (not storing into a reference)
// so that it doesn't get removed later for being useless.
emit(Osalloc, Kl, R, getcon(SHADOW_SPACE_SIZE, func), R);
}
return instr_past_args;
}
static void lower_block_return(Fn* func, Blk* block) {
int jmp_type = block->jmp.type;
if (!isret(jmp_type) || jmp_type == Jret0) {
return;
}
// Save the argument, and set the block to be a void return because once it's
// lowered it's handled by the the register/stack manipulation.
Ref ret_arg = block->jmp.arg;
block->jmp.type = Jret0;
RegisterUsage reg_usage = {0};
if (jmp_type == Jretc) {
Typ* type = &typ[func->retty];
if (type_is_by_copy(type)) {
assert(rtype(func->retr) == RTmp);
emit(Ocopy, Kl, TMP(RAX), func->retr, R);
emit(Oblit1, 0, R, INT(type->size), R);
emit(Oblit0, 0, R, ret_arg, func->retr);
} else {
emit(Oload, Kl, TMP(RAX), ret_arg, R);
}
reg_usage.rax_returned = true;
} else {
int k = jmp_type - Jretw;
if (is_integer_type(k)) {
emit(Ocopy, k, TMP(RAX), ret_arg, R);
reg_usage.rax_returned = true;
} else {
emit(Ocopy, k, TMP(XMM0), ret_arg, R);
reg_usage.xmm0_returned = true;
}
}
block->jmp.arg = CALL(register_usage_to_call_arg_value(reg_usage));
}
static void lower_vastart(Fn* func,
RegisterUsage* param_reg_usage,
Ref valist) {
assert(func->vararg);
// In varargs functions:
// 1. the int registers are already dumped to the shadow stack space;
// 2. any parameters passed in floating point registers have
// been duplicated to the integer registers
// 3. we ensure (later) that for varargs functions we're always using an rbp
// frame pointer.
// So, the ... argument is just indexed past rbp by the number of named values
// that were actually passed.
Ref offset = newtmp("abi.vastart", Kl, func);
emit(Ostorel, 0, R, offset, valist);
// *8 for sizeof(u64), +16 because the return address and rbp have been pushed
// by the time we get to the body of the function.
emit(Oadd, Kl, offset, TMP(RBP),
getcon(param_reg_usage->num_named_args_passed * 8 + 16, func));
}
static void lower_vaarg(Fn* func, Ins* vaarg_instr) {
// va_list is just a void** on winx64, so load the pointer, then load the
// argument from that pointer, then increment the pointer to the next arg.
// (All emitted backwards as usual.)
Ref inc = newtmp("abi.vaarg.inc", Kl, func);
Ref ptr = newtmp("abi.vaarg.ptr", Kl, func);
emit(Ostorel, 0, R, inc, vaarg_instr->arg[0]);
emit(Oadd, Kl, inc, ptr, getcon(8, func));
emit(Oload, vaarg_instr->cls, vaarg_instr->to, ptr, R);
emit(Oload, Kl, ptr, vaarg_instr->arg[0], R);
}
static void lower_args_for_block(Fn* func,
Blk* block,
RegisterUsage* param_reg_usage,
ExtraAlloc** pextra_alloc) {
// global temporary buffer used by emit. Reset to the end, and predecremented
// when adding to it.
curi = &insb[NIns];
lower_block_return(func, block);
if (block->nins) {
// Work backwards through the instructions, either copying them unchanged,
// or modifying as necessary.
for (Ins* instr = &block->ins[block->nins - 1]; instr >= block->ins;) {
switch (instr->op) {
case Ocall:
instr = lower_call(func, block, instr, pextra_alloc);
break;
case Ovastart:
lower_vastart(func, param_reg_usage, instr->arg[0]);
--instr;
break;
case Ovaarg:
lower_vaarg(func, instr);
--instr;
break;
case Oarg:
case Oargc:
die("unreachable");
default:
emiti(*instr);
--instr;
break;
}
}
}
// This it the start block, which is processed last. Add any allocas that
// other blocks needed.
bool is_start_block = block == func->start;
if (is_start_block) {
for (ExtraAlloc* ea = *pextra_alloc; ea; ea = ea->link) {
emiti(ea->instr);
}
}
// emit/emiti add instructions from the end to the beginning of the temporary
// global buffer. dup the final version into the final block storage.
block->nins = &insb[NIns] - curi;
idup(block, curi, block->nins);
}
static Ins* find_end_of_func_parameters(Blk* start_block) {
Ins* i;
for (i = start_block->ins; i < &start_block->ins[start_block->nins]; ++i) {
if (!ispar(i->op)) {
break;
}
}
return i;
}
// Copy from registers/stack into values.
static RegisterUsage lower_func_parameters(Fn* func) {
// This is half-open, so end points after the last Opar.
Blk* start_block = func->start;
Ins* start_of_params = start_block->ins;
Ins* end_of_params = find_end_of_func_parameters(start_block);
size_t num_params = end_of_params - start_of_params;
ArgClass* arg_classes = alloc(num_params * sizeof(ArgClass));
ArgClass arg_ret = {0};
// global temporary buffer used by emit. Reset to the end, and predecremented
// when adding to it.
curi = &insb[NIns];
int reg_counter = 0;
RegisterUsage reg_usage = {0};
if (func->retty >= 0) {
bool by_copy = type_is_by_copy(&typ[func->retty]);
if (by_copy) {
assign_register_or_stack(&reg_usage, &arg_ret, /*is_float=*/false,
by_copy);
Ref ret_ref = newtmp("abi.ret", Kl, func);
emit(Ocopy, Kl, ret_ref, TMP(RCX), R);
func->retr = ret_ref;
++reg_counter;
}
}
Ref env = R;
classify_arguments(&reg_usage, start_of_params, end_of_params, arg_classes,
&env);
func->reg = amd64_winabi_argregs(
CALL(register_usage_to_call_arg_value(reg_usage)), NULL);
// Copy from the registers or stack slots into the named parameters. Depending
// on how they're passed, they either need to be copied or loaded.
ArgClass* arg = arg_classes;
uint slot_offset = SHADOW_SPACE_SIZE / 4 + 4;
for (Ins* instr = start_of_params; instr < end_of_params; ++instr, ++arg) {
switch (arg->style) {
case APS_Register: {
Ref from = register_for_arg(arg->cls, reg_counter++);
// If it's a struct at the IL level, we need to copy the register into
// an alloca so we have something to point at (same for InlineOnStack).
if (instr->op == Oparc) {
arg->ref = newtmp("abi", Kl, func);
emit(Ostorel, 0, R, arg->ref, instr->to);
emit(Ocopy, instr->cls, arg->ref, from, R);
emit(Oalloc8, Kl, instr->to, getcon(arg->size, func), R);
} else {
emit(Ocopy, instr->cls, instr->to, from, R);
}
break;
}
case APS_InlineOnStack:
if (instr->op == Oparc) {
arg->ref = newtmp("abi", Kl, func);
emit(Ostorel, 0, R, arg->ref, instr->to);
emit(Ocopy, instr->cls, arg->ref, SLOT(-slot_offset), R);
emit(Oalloc8, Kl, instr->to, getcon(arg->size, func), R);
} else {
emit(Ocopy, Kl, instr->to, SLOT(-slot_offset), R);
}
slot_offset += 2;
break;
case APS_CopyAndPointerOnStack:
emit(Oload, Kl, instr->to, SLOT(-slot_offset), R);
slot_offset += 2;
break;
case APS_CopyAndPointerInRegister: {
// Because this has to be a copy (that we own), it is sufficient to just
// copy the register to the target.
Ref from = register_for_arg(Kl, reg_counter++);
emit(Ocopy, Kl, instr->to, from, R);
break;
}
case APS_EnvTag:
break;
case APS_VarargsTag:
case APS_Invalid:
die("unreachable");
}
}
// If there was an `env`, it was passed in RAX, so copy it into the env ref.
if (!req(R, env)) {
emit(Ocopy, Kl, env, TMP(RAX), R);
}
int num_created_instrs = &insb[NIns] - curi;
int num_other_after_instrs = (int)(start_block->nins - num_params);
int new_total_instrs = num_other_after_instrs + num_created_instrs;
Ins* new_instrs = vnew(new_total_instrs, sizeof(Ins), PFn);
Ins* instr_p = icpy(new_instrs, curi, num_created_instrs);
icpy(instr_p, end_of_params, num_other_after_instrs);
start_block->nins = new_total_instrs;
start_block->ins = new_instrs;
return reg_usage;
}
// The main job of this function is to lower generic instructions into the
// specific details of how arguments are passed, and parameters are
// interpreted for win x64. A useful reference is
// https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention .
//
// Some of the major differences from SysV if you're comparing the code
// (non-exhaustive):
// - only 4 int and 4 float regs are used
// - when an int register is assigned a value, its associated float register is
// left unused (and vice versa). i.e. there's only one counter as you assign
// arguments to registers.
// - any structs that aren't 1/2/4/8 bytes in size are passed by pointer, not
// by copying them into the stack. So e.g. if you pass something like
// `struct { void*, int64_t }` by value, it first needs to be copied to
// another alloca (in order to maintain value semantics at the language
// level), then the pointer to that copy is treated as a regular integer
// argument (which then itself may *also* be copied to the stack in the case
// there's no integer register remaining.)
// - when calling a varargs functions, floating point values must be duplicated
// integer registers. Along with the above restrictions, this makes varargs
// handling simpler for the callee than SysV.
void amd64_winabi_abi(Fn* func) {
// The first thing to do is lower incoming parameters to this function.
RegisterUsage param_reg_usage = lower_func_parameters(func);
// This is the second larger part of the job. We walk all blocks, and rewrite
// instructions returns, calls, and handling of varargs into their win x64
// specific versions. Any other instructions are just passed through unchanged
// by using `emiti`.
// Skip over the entry block, and do it at the end so that our later
// modifications can add allocations to the start block. In particular, we
// need to add stack allocas for copies when structs are passed or returned by
// value.
ExtraAlloc* extra_alloc = NULL;
for (Blk* block = func->start->link; block; block = block->link) {
lower_args_for_block(func, block, &param_reg_usage, &extra_alloc);
}
lower_args_for_block(func, func->start, &param_reg_usage, &extra_alloc);
if (debug['A']) {
fprintf(stderr, "\n> After ABI lowering:\n");
printfn(func, stderr);
}
}

View File

@ -429,7 +429,7 @@ selcall(Fn *fn, Ins *i0, Ins *i1, Insl **ilp)
for (i=i0, c=ca; i<i1; i++, c++) { for (i=i0, c=ca; i<i1; i++, c++) {
if ((c->class & Cstk) != 0) if ((c->class & Cstk) != 0)
continue; continue;
if (i->op == Oarg || i->op == Oarge) if (i->op == Oarg || i->op == Oarge || isargbh(i->op))
emit(Ocopy, *c->cls, TMP(*c->reg), i->arg[0], R); emit(Ocopy, *c->cls, TMP(*c->reg), i->arg[0], R);
if (i->op == Oargc) if (i->op == Oargc)
ldregs(c->reg, c->cls, c->nreg, i->arg[1], fn); ldregs(c->reg, c->cls, c->nreg, i->arg[1], fn);

View File

@ -10,24 +10,24 @@ struct E {
}; };
#define CMP(X) \ #define CMP(X) \
X(Cieq, "eq") \ X(Cieq, "eq", "ne") \
X(Cine, "ne") \ X(Cine, "ne", "eq") \
X(Cisge, "ge") \ X(Cisge, "ge", "lt") \
X(Cisgt, "gt") \ X(Cisgt, "gt", "le") \
X(Cisle, "le") \ X(Cisle, "le", "gt") \
X(Cislt, "lt") \ X(Cislt, "lt", "ge") \
X(Ciuge, "cs") \ X(Ciuge, "cs", "cc") \
X(Ciugt, "hi") \ X(Ciugt, "hi", "ls") \
X(Ciule, "ls") \ X(Ciule, "ls", "hi") \
X(Ciult, "cc") \ X(Ciult, "cc", "cs") \
X(NCmpI+Cfeq, "eq") \ X(NCmpI+Cfeq, "eq", "ne") \
X(NCmpI+Cfge, "ge") \ X(NCmpI+Cfge, "ge", "lt") \
X(NCmpI+Cfgt, "gt") \ X(NCmpI+Cfgt, "gt", "le") \
X(NCmpI+Cfle, "ls") \ X(NCmpI+Cfle, "ls", "hi") \
X(NCmpI+Cflt, "mi") \ X(NCmpI+Cflt, "mi", "pl") \
X(NCmpI+Cfne, "ne") \ X(NCmpI+Cfne, "ne", "eq") \
X(NCmpI+Cfo, "vc") \ X(NCmpI+Cfo, "vc", "vs") \
X(NCmpI+Cfuo, "vs") X(NCmpI+Cfuo, "vs", "vc")
enum { enum {
Ki = -1, /* matches Kw and Kl */ Ki = -1, /* matches Kw and Kl */
@ -102,13 +102,17 @@ static struct {
{ Oacmn, Ki, "cmn %0, %1" }, { Oacmn, Ki, "cmn %0, %1" },
{ Oafcmp, Ka, "fcmpe %0, %1" }, { Oafcmp, Ka, "fcmpe %0, %1" },
#define X(c, str) \ #define X(c, str, _) \
{ Oflag+c, Ki, "cset %=, " str }, { Oflag+c, Ki, "cset %=, " str },
CMP(X) CMP(X)
#undef X #undef X
{ NOp, 0, 0 } { NOp, 0, 0 }
}; };
enum {
V31 = 0x1fffffff, /* local name for V31 */
};
static char * static char *
rname(int r, int k) rname(int r, int k)
{ {
@ -132,6 +136,12 @@ rname(int r, int k)
case Kx: case Kx:
case Kd: sprintf(buf, "d%d", r-V0); break; case Kd: sprintf(buf, "d%d", r-V0); break;
} }
else if (r == V31)
switch (k) {
default: die("invalid class");
case Ks: sprintf(buf, "s31"); break;
case Kd: sprintf(buf, "d31"); break;
}
else else
die("invalid register"); die("invalid register");
return buf; return buf;
@ -197,12 +207,12 @@ emitf(char *s, Ins *i, E *e)
if (KBASE(k) == 0) if (KBASE(k) == 0)
fputs(rname(IP1, k), e->f); fputs(rname(IP1, k), e->f);
else else
fputs(k==Ks ? "s31" : "d31", e->f); fputs(rname(V31, k), e->f);
break; break;
case '=': case '=':
case '0': case '0':
r = c == '=' ? i->to : i->arg[0]; r = c == '=' ? i->to : i->arg[0];
assert(isreg(r)); assert(isreg(r) || req(r, TMP(V31)));
fputs(rname(r.val, k), e->f); fputs(rname(r.val, k), e->f);
break; break;
case '1': case '1':
@ -268,6 +278,10 @@ loadaddr(Con *c, char *rn, E *e)
s = "\tadrp\tR, SO\n" s = "\tadrp\tR, SO\n"
"\tadd\tR, R, #:lo12:SO\n"; "\tadd\tR, R, #:lo12:SO\n";
break; break;
case SExtThr:
if (!T.apple)
die("extern thread unavailable on arm64");
/* fall through */
case SThr: case SThr:
if (T.apple) if (T.apple)
s = "\tadrp\tR, S@tlvppage\n" s = "\tadrp\tR, S@tlvppage\n"
@ -277,6 +291,14 @@ loadaddr(Con *c, char *rn, E *e)
"\tadd\tR, R, #:tprel_hi12:SO, lsl #12\n" "\tadd\tR, R, #:tprel_hi12:SO, lsl #12\n"
"\tadd\tR, R, #:tprel_lo12_nc:SO\n"; "\tadd\tR, R, #:tprel_lo12_nc:SO\n";
break; break;
case SExt:
if (T.apple)
s = "\tadrp\tR, S@gotpageO\n"
"\tldr\tR, [R, S@gotpageoffO]\n";
else
s = "\tadrp\tR, :got:SO\n"
"\tldr\tR, [R, #:got_lo12:SO]\n";
break;
} }
l = str(c->sym.id); l = str(c->sym.id);
@ -335,8 +357,8 @@ loadcon(Con *c, int r, int k, E *e)
static void emitins(Ins *, E *); static void emitins(Ins *, E *);
static void static int
fixarg(Ref *pr, int sz, E *e) fixarg(Ref *pr, int sz, int t, E *e)
{ {
Ins *i; Ins *i;
Ref r; Ref r;
@ -346,11 +368,14 @@ fixarg(Ref *pr, int sz, E *e)
if (rtype(r) == RSlot) { if (rtype(r) == RSlot) {
s = slot(r, e); s = slot(r, e);
if (s > sz * 4095u) { if (s > sz * 4095u) {
i = &(Ins){Oaddr, Kl, TMP(IP1), {r}}; if (t < 0)
return 1;
i = &(Ins){Oaddr, Kl, TMP(t), {r}};
emitins(i, e); emitins(i, e);
*pr = TMP(IP1); *pr = TMP(t);
} }
} }
return 0;
} }
static void static void
@ -358,16 +383,28 @@ emitins(Ins *i, E *e)
{ {
char *l, *p, *rn; char *l, *p, *rn;
uint64_t s; uint64_t s;
int o; int o, t;
Ref r; Ref r;
Con *c; Con *c;
switch (i->op) { switch (i->op) {
default: default:
if (isload(i->op)) if (isload(i->op))
fixarg(&i->arg[0], loadsz(i), e); fixarg(&i->arg[0], loadsz(i), IP1, e);
if (isstore(i->op)) if (isstore(i->op)) {
fixarg(&i->arg[1], storesz(i), e); t = T.apple ? -1 : R18;
if (fixarg(&i->arg[1], storesz(i), t, e)) {
if (req(i->arg[0], TMP(IP1))) {
fprintf(e->f,
"\tfmov\t%c31, %c17\n",
"ds"[i->cls == Kw],
"xw"[i->cls == Kw]);
i->arg[0] = TMP(V31);
i->op = Ostores + (i->cls-Kw);
}
fixarg(&i->arg[1], storesz(i), IP1, e);
}
}
Table: Table:
/* most instructions are just pulled out of /* most instructions are just pulled out of
* the table omap[], some special cases are * the table omap[], some special cases are
@ -443,7 +480,7 @@ emitins(Ins *i, E *e)
goto Table; goto Table;
c = &e->fn->con[i->arg[0].val]; c = &e->fn->con[i->arg[0].val];
if (c->type != CAddr if (c->type != CAddr
|| c->sym.type != SGlo || (c->sym.type & SThr)
|| c->bits.i) || c->bits.i)
die("invalid call argument"); die("invalid call argument");
l = str(c->sym.id); l = str(c->sym.id);
@ -507,8 +544,8 @@ framelayout(E *e)
void void
arm64_emitfn(Fn *fn, FILE *out) arm64_emitfn(Fn *fn, FILE *out)
{ {
static char *ctoa[] = { static char *ctoa[][2] = {
#define X(c, s) [c] = s, #define X(c, s, n) [c] = {s, n},
CMP(X) CMP(X)
#undef X #undef X
}; };
@ -639,11 +676,12 @@ arm64_emitfn(Fn *fn, FILE *out)
t = b->s1; t = b->s1;
b->s1 = b->s2; b->s1 = b->s2;
b->s2 = t; b->s2 = t;
n = 0;
} else } else
c = cmpneg(c); n = 1;
fprintf(e->f, fprintf(e->f,
"\tb%s\t%s%d\n", "\tb%s\t%s%d\n",
ctoa[c], T.asloc, id0+b->s2->id ctoa[c][n], T.asloc, id0+b->s2->id
); );
goto Jmp; goto Jmp;
} }

View File

@ -80,7 +80,7 @@ fixarg(Ref *pr, int k, int phi, Fn *fn)
c = &fn->con[r0.val]; c = &fn->con[r0.val];
if (T.apple if (T.apple
&& c->type == CAddr && c->type == CAddr
&& c->sym.type == SThr) { && (c->sym.type & SThr)) {
r1 = newtmp("isel", Kl, fn); r1 = newtmp("isel", Kl, fn);
*pr = r1; *pr = r1;
if (c->bits.i) { if (c->bits.i) {
@ -109,7 +109,7 @@ fixarg(Ref *pr, int k, int phi, Fn *fn)
if (KBASE(k) == 0) { if (KBASE(k) == 0) {
emit(Ocopy, k, r1, r0, R); emit(Ocopy, k, r1, r0, R);
} else { } else {
n = stashbits(&c->bits, KWIDE(k) ? 8 : 4); n = stashbits(c->bits.i, KWIDE(k) ? 8 : 4);
vgrow(&fn->con, ++fn->ncon); vgrow(&fn->con, ++fn->ncon);
c = &fn->con[fn->ncon-1]; c = &fn->con[fn->ncon-1];
sprintf(buf, "\"%sfp%d\"", T.asloc, n); sprintf(buf, "\"%sfp%d\"", T.asloc, n);

View File

@ -40,6 +40,7 @@ arm64_memargs(int op)
.isel = arm64_isel, \ .isel = arm64_isel, \
.abi1 = arm64_abi, \ .abi1 = arm64_abi, \
.emitfn = arm64_emitfn, \ .emitfn = arm64_emitfn, \
.cansel = 0, \
Target T_arm64 = { Target T_arm64 = {
.name = "arm64", .name = "arm64",

177
cfg.c
View File

@ -16,19 +16,22 @@ newblk()
static void static void
fixphis(Fn *f) fixphis(Fn *f)
{ {
Blk *b; Blk *b, *bp;
Phi *p; Phi *p;
uint n, n0; uint n, n0;
for (b=f->start; b; b=b->link) { for (b=f->start; b; b=b->link) {
assert(b->id < f->nblk); assert(b->id < f->nblk);
for (p=b->phi; p; p=p->link) { for (p=b->phi; p; p=p->link) {
for (n=n0=0; n<p->narg; n++) for (n=n0=0; n<p->narg; n++) {
if (p->blk[n]->id != -1u) { bp = p->blk[n];
p->blk[n0] = p->blk[n]; if (bp->id != -1u)
if (bp->s1 == b || bp->s2 == b) {
p->blk[n0] = bp;
p->arg[n0] = p->arg[n]; p->arg[n0] = p->arg[n];
n0++; n0++;
} }
}
assert(n0 > 0); assert(n0 > 0);
p->narg = n0; p->narg = n0;
} }
@ -396,3 +399,169 @@ reachesnotvia(Fn *fn, Blk *b, Blk *to, Blk *excl)
excl->visit = 1; excl->visit = 1;
return reaches(fn, b, to); return reaches(fn, b, to);
} }
int
ifgraph(Blk *ifb, Blk **pthenb, Blk **pelseb, Blk **pjoinb)
{
Blk *s1, *s2, **t;
if (ifb->jmp.type != Jjnz)
return 0;
s1 = ifb->s1;
s2 = ifb->s2;
if (s1->id > s2->id) {
s1 = ifb->s2;
s2 = ifb->s1;
t = pthenb;
pthenb = pelseb;
pelseb = t;
}
if (s1 == s2)
return 0;
if (s1->jmp.type != Jjmp || s1->npred != 1)
return 0;
if (s1->s1 == s2) {
/* if-then / if-else */
if (s2->npred != 2)
return 0;
*pthenb = s1;
*pelseb = ifb;
*pjoinb = s2;
return 1;
}
if (s2->jmp.type != Jjmp || s2->npred != 1)
return 0;
if (s1->s1 != s2->s1 || s1->s1->npred != 2)
return 0;
assert(s1->s1 != ifb);
*pthenb = s1;
*pelseb = s2;
*pjoinb = s1->s1;
return 1;
}
typedef struct Jmp Jmp;
struct Jmp {
int type;
Ref arg;
Blk *s1, *s2;
};
static int
jmpeq(Jmp *a, Jmp *b)
{
return a->type == b->type && req(a->arg, b->arg)
&& a->s1 == b->s1 && a->s2 == b->s2;
}
static int
jmpnophi(Jmp *j)
{
if (j->s1 && j->s1->phi)
return 0;
if (j->s2 && j->s2->phi)
return 0;
return 1;
}
/* require cfg rpo, breaks use */
void
simplcfg(Fn *fn)
{
Ins cpy, *i;
Blk *b, *bb, **pb;
Jmp *jmp, *j, *jj;
Phi *p;
int *empty, done;
uint n;
if (debug['C']) {
fprintf(stderr, "\n> Before CFG simplification:\n");
printfn(fn, stderr);
}
cpy = (Ins){.op = Ocopy};
for (b=fn->start; b; b=b->link)
if (b->npred == 1) {
bb = b->pred[0];
for (p=b->phi; p; p=p->link) {
cpy.cls = p->cls;
cpy.to = p->to;
cpy.arg[0] = phiarg(p, bb);
addins(&bb->ins, &bb->nins, &cpy);
}
b->phi = 0;
}
jmp = emalloc(fn->nblk * sizeof jmp[0]);
empty = emalloc(fn->nblk * sizeof empty[0]);
for (b=fn->start; b; b=b->link) {
jmp[b->id].type = b->jmp.type;
jmp[b->id].arg = b->jmp.arg;
jmp[b->id].s1 = b->s1;
jmp[b->id].s2 = b->s2;
empty[b->id] = !b->phi;
for (i=b->ins; i<&b->ins[b->nins]; i++)
if (i->op != Onop && i->op != Odbgloc) {
empty[b->id] = 0;
break;
}
}
do {
done = 1;
for (b=fn->start; b; b=b->link) {
if (b->id == -1u)
continue;
j = &jmp[b->id];
if (j->type == Jjmp && j->s1->npred == 1) {
assert(!j->s1->phi);
addbins(&b->ins, &b->nins, j->s1);
empty[b->id] &= empty[j->s1->id];
jj = &jmp[j->s1->id];
pb = (Blk*[]){jj->s1, jj->s2, 0};
for (; (bb=*pb); pb++)
for (p=bb->phi; p; p=p->link) {
n = phiargn(p, j->s1);
p->blk[n] = b;
}
j->s1->id = -1u;
*j = *jj;
done = 0;
}
else if (j->type == Jjnz
&& empty[j->s1->id] && empty[j->s2->id]
&& jmpeq(&jmp[j->s1->id], &jmp[j->s2->id])
&& jmpnophi(&jmp[j->s1->id])) {
*j = jmp[j->s1->id];
done = 0;
}
}
} while (!done);
for (b=fn->start; b; b=b->link)
if (b->id != -1u) {
j = &jmp[b->id];
b->jmp.type = j->type;
b->jmp.arg = j->arg;
b->s1 = j->s1;
b->s2 = j->s2;
assert(!j->s1 || j->s1->id != -1u);
assert(!j->s2 || j->s2->id != -1u);
}
fillcfg(fn);
free(empty);
free(jmp);
if (debug['C']) {
fprintf(stderr, "\n> After CFG simplification:\n");
printfn(fn, stderr);
}
}

35
copy.c
View File

@ -41,9 +41,8 @@ bitwidth(uint64_t v)
return n+v; return n+v;
} }
/* no more than w bits are used */
static int static int
usewidthle(Fn *fn, Ref r, int w) uwl(Fn *fn, Ref r, int w)
{ {
Ext e; Ext e;
Tmp *t; Tmp *t;
@ -52,7 +51,6 @@ usewidthle(Fn *fn, Ref r, int w)
Ins *i; Ins *i;
Ref rc; Ref rc;
int64_t v; int64_t v;
int b;
assert(rtype(r) == RTmp); assert(rtype(r) == RTmp);
t = &fn->tmp[r.val]; t = &fn->tmp[r.val];
@ -60,24 +58,28 @@ usewidthle(Fn *fn, Ref r, int w)
switch (u->type) { switch (u->type) {
case UPhi: case UPhi:
p = u->u.phi; p = u->u.phi;
if (p->visit) /* during gvn, phi nodes may be
* replaced by other temps; in
* this case, the replaced phi
* uses are added to the
* replacement temp uses and
* Phi.to is set to R */
if (p->visit || req(p->to, R))
continue; continue;
p->visit = 1; p->visit = 1;
b = usewidthle(fn, p->to, w); if (uwl(fn, p->to, w))
p->visit = 0;
if (b)
continue; continue;
break; break;
case UIns: case UIns:
i = u->u.ins; i = u->u.ins;
assert(i != 0); assert(i != 0);
if (i->op == Ocopy) if (i->op == Ocopy)
if (usewidthle(fn, i->to, w)) if (uwl(fn, i->to, w))
continue; continue;
if (ext(i, &e)) { if (ext(i, &e)) {
if (e.usew <= w) if (e.usew <= w)
continue; continue;
if (usewidthle(fn, i->to, w)) if (uwl(fn, i->to, w))
continue; continue;
} }
if (i->op == Oand) { if (i->op == Oand) {
@ -101,6 +103,21 @@ usewidthle(Fn *fn, Ref r, int w)
return 1; return 1;
} }
/* no more than w bits are used */
static int
usewidthle(Fn *fn, Ref r, int w)
{
Blk *b;
Phi *p;
int ret;
ret = uwl(fn, r, w);
for (b=fn->start; b; b=b->link)
for (p=b->phi; p; p=p->link)
p->visit = 0;
return ret;
}
static int static int
min(int v1, int v2) min(int v1, int v2)
{ {

View File

@ -181,6 +181,8 @@ by zero-extension, or by sign-extension.
DYNCONST := DYNCONST :=
CONST CONST
| 'thread' $IDENT # Thread-local symbol | 'thread' $IDENT # Thread-local symbol
| 'extern' $IDENT # Extern symbol (GOT)
| 'extern' 'thread' $IDENT # Extern thread-local (initial-exec)
VAL := VAL :=
DYNCONST DYNCONST
@ -225,6 +227,15 @@ When the `thread` keyword prefixes a symbol name, the
symbol's numeric value is resolved at runtime in the symbol's numeric value is resolved at runtime in the
thread-local storage. thread-local storage.
When the `extern` keyword prefixes a symbol name, the
symbol is accessed indirectly through a table edited
by the dynamic linker (e.g., GOT/PLT). This enables
PIE/PIC code generation. When `extern` is combined
with `thread`, the symbol is accessed using the
initial-exec TLS model, suitable for thread-local
variables defined in shared objects available at
startup time (i.e., not loaded through dlopen).
Vals are used as arguments in regular, phi, and jump Vals are used as arguments in regular, phi, and jump
instructions within function definitions. They are instructions within function definitions. They are
either constants or function-scope temporaries. either constants or function-scope temporaries.
@ -976,6 +987,7 @@ is possible to conservatively use the maximum size and
alignment required by all the targets. alignment required by all the targets.
type :valist = align 8 { 24 } # For amd64_sysv type :valist = align 8 { 24 } # For amd64_sysv
type :valist = align 8 { 8 } # For amd64_win
type :valist = align 8 { 32 } # For arm64 type :valist = align 8 { 32 } # For arm64
type :valist = align 8 { 8 } # For rv64 type :valist = align 8 { 8 } # For rv64

15
doc/native_win.txt Normal file
View File

@ -0,0 +1,15 @@
There is an experimental amd64_win (native Windows ABI and calling
convention).
In tree, this is currently only tested via cross-compilation from a
Linux host, and using wine to run the tests.
You'll need something like:
sudo apt install mingw64-w64 dos2unix wine
and then
make check-amd64_win
should pass.

69
emit.c
View File

@ -61,11 +61,14 @@ emitfnlnk(char *n, Lnk *l, FILE *f)
void void
emitdat(Dat *d, FILE *f) emitdat(Dat *d, FILE *f)
{ {
static char *dtoa[] = { static struct {
[DB] = "\t.byte", char decl[8];
[DH] = "\t.short", int64_t mask;
[DW] = "\t.int", } di[] = {
[DL] = "\t.quad" [DB] = {"\t.byte", 0xffL},
[DH] = {"\t.short", 0xffffL},
[DW] = {"\t.int", 0xffffffffL},
[DL] = {"\t.quad", -1L},
}; };
static int64_t zero; static int64_t zero;
char *p; char *p;
@ -111,12 +114,13 @@ emitdat(Dat *d, FILE *f)
else if (d->isref) { else if (d->isref) {
p = d->u.ref.name[0] == '"' ? "" : T.assym; p = d->u.ref.name[0] == '"' ? "" : T.assym;
fprintf(f, "%s %s%s%+"PRId64"\n", fprintf(f, "%s %s%s%+"PRId64"\n",
dtoa[d->type], p, d->u.ref.name, di[d->type].decl, p, d->u.ref.name,
d->u.ref.off); d->u.ref.off);
} }
else { else {
fprintf(f, "%s %"PRId64"\n", fprintf(f, "%s %"PRId64"\n",
dtoa[d->type], d->u.num); di[d->type].decl,
d->u.num & di[d->type].mask);
} }
break; break;
} }
@ -125,7 +129,7 @@ emitdat(Dat *d, FILE *f)
typedef struct Asmbits Asmbits; typedef struct Asmbits Asmbits;
struct Asmbits { struct Asmbits {
char bits[16]; bits n;
int size; int size;
Asmbits *link; Asmbits *link;
}; };
@ -133,18 +137,17 @@ struct Asmbits {
static Asmbits *stash; static Asmbits *stash;
int int
stashbits(void *bits, int size) stashbits(bits n, int size)
{ {
Asmbits **pb, *b; Asmbits **pb, *b;
int i; int i;
assert(size == 4 || size == 8 || size == 16); assert(size == 4 || size == 8 || size == 16);
for (pb=&stash, i=0; (b=*pb); pb=&b->link, i++) for (pb=&stash, i=0; (b=*pb); pb=&b->link, i++)
if (size <= b->size) if (size <= b->size && b->n == n)
if (memcmp(bits, b->bits, size) == 0)
return i; return i;
b = emalloc(sizeof *b); b = emalloc(sizeof *b);
memcpy(b->bits, bits, size); b->n = n;
b->size = size; b->size = size;
b->link = 0; b->link = 0;
*pb = b; *pb = b;
@ -155,9 +158,8 @@ static void
emitfin(FILE *f, char *sec[3]) emitfin(FILE *f, char *sec[3])
{ {
Asmbits *b; Asmbits *b;
char *p;
int lg, i; int lg, i;
double d; union { int32_t i; float f; } u;
if (!stash) if (!stash)
return; return;
@ -171,17 +173,24 @@ emitfin(FILE *f, char *sec[3])
"%sfp%d:", "%sfp%d:",
sec[lg-2], lg, T.asloc, i sec[lg-2], lg, T.asloc, i
); );
for (p=b->bits; p<&b->bits[b->size]; p+=4) if (lg == 4)
fprintf(f, "\n\t.int %"PRId32, fprintf(f,
*(int32_t *)p); "\n\t.quad %"PRId64
if (lg <= 3) { "\n\t.quad 0\n\n",
if (lg == 2) (int64_t)b->n);
d = *(float *)b->bits; else if (lg == 3)
else fprintf(f,
d = *(double *)b->bits; "\n\t.quad %"PRId64
fprintf(f, " /* %f */\n\n", d); " /* %f */\n\n",
} else (int64_t)b->n,
fprintf(f, "\n\n"); *(double *)&b->n);
else if (lg == 2) {
u.i = b->n;
fprintf(f,
"\n\t.int %"PRId32
" /* %f */\n\n",
u.i, (double)u.f);
}
} }
} }
while ((b=stash)) { while ((b=stash)) {
@ -212,12 +221,20 @@ macho_emitfin(FILE *f)
static char *sec[3] = { static char *sec[3] = {
"__TEXT,__literal4,4byte_literals", "__TEXT,__literal4,4byte_literals",
"__TEXT,__literal8,8byte_literals", "__TEXT,__literal8,8byte_literals",
".abort \"unreachable\"", "__TEXT,__literal16,16byte_literals",
}; };
emitfin(f, sec); emitfin(f, sec);
} }
void
pe_emitfin(FILE *f)
{
static char *sec[3] = { ".rodata", ".rodata", ".rodata" };
emitfin(f, sec);
}
static uint32_t *file; static uint32_t *file;
static uint nfile; static uint nfile;
static uint curfile; static uint curfile;

8
gvn.c
View File

@ -43,11 +43,11 @@ static uint gvntbln;
static Ins * static Ins *
gvndup(Ins *i, int insert) gvndup(Ins *i, int insert)
{ {
uint idx, n; uint idx;
Ins *ii; Ins *ii;
idx = ihash(i) % gvntbln; idx = ihash(i) % gvntbln;
for (n=1;; n++) { for (;;) {
ii = gvntbl[idx]; ii = gvntbl[idx];
if (!ii) if (!ii)
break; break;
@ -247,6 +247,10 @@ dedupins(Fn *fn, Blk *b, Ins *i)
if (i->op == Onop || pinned(i)) if (i->op == Onop || pinned(i))
return; return;
/* when sel instructions are inserted
* before gvn, we may want to optimize
* them here */
assert(i->op != Osel0);
assert(!req(i->to, R)); assert(!req(i->to, R));
assoccon(fn, b, i); assoccon(fn, b, i);

121
ifopt.c Normal file
View File

@ -0,0 +1,121 @@
#include "all.h"
enum {
MaxIns = 2,
MaxPhis = 2,
};
static int
okbranch(Blk *b)
{
Ins *i;
int n;
n = 0;
for (i=b->ins; i<&b->ins[b->nins]; i++)
if (i->op != Odbgloc) {
if (pinned(i))
return 0;
if (i->op != Onop)
n++;
}
return n <= MaxIns;
}
static int
okjoin(Blk *b)
{
Phi *p;
int n;
n = 0;
for (p=b->phi; p; p=p->link) {
if (KBASE(p->cls) != 0)
return 0;
n++;
}
return n <= MaxPhis;
}
static int
okgraph(Blk *ifb, Blk *thenb, Blk *elseb, Blk *joinb)
{
if (joinb->npred != 2 || !okjoin(joinb))
return 0;
assert(thenb != elseb);
if (thenb != ifb && !okbranch(thenb))
return 0;
if (elseb != ifb && !okbranch(elseb))
return 0;
return 1;
}
static void
convert(Blk *ifb, Blk *thenb, Blk *elseb, Blk *joinb)
{
Ins *ins, sel;
Phi *p;
uint nins;
ins = vnew(0, sizeof ins[0], PHeap);
nins = 0;
addbins(&ins, &nins, ifb);
if (thenb != ifb)
addbins(&ins, &nins, thenb);
if (elseb != ifb)
addbins(&ins, &nins, elseb);
assert(joinb->npred == 2);
if (joinb->phi) {
sel = (Ins){
.op = Osel0, .cls = Kw,
.arg = {ifb->jmp.arg},
};
addins(&ins, &nins, &sel);
}
sel = (Ins){.op = Osel1};
for (p=joinb->phi; p; p=p->link) {
sel.to = p->to;
sel.cls = p->cls;
sel.arg[0] = phiarg(p, thenb);
sel.arg[1] = phiarg(p, elseb);
addins(&ins, &nins, &sel);
}
idup(ifb, ins, nins);
ifb->jmp.type = Jjmp;
ifb->jmp.arg = R;
ifb->s1 = joinb;
ifb->s2 = 0;
joinb->npred = 1;
joinb->pred[0] = ifb;
joinb->phi = 0;
vfree(ins);
}
/* eliminate if-then[-else] graphlets
* using sel instructions
* needs rpo pred use; breaks cfg use
*/
void
ifconvert(Fn *fn)
{
Blk *ifb, *thenb, *elseb, *joinb;
if (debug['K'])
fputs("\n> If-conversion:\n", stderr);
for (ifb=fn->start; ifb; ifb=ifb->link)
if (ifgraph(ifb, &thenb, &elseb, &joinb))
if (okgraph(ifb, thenb, elseb, joinb)) {
if (debug['K'])
fprintf(stderr,
" @%s -> @%s, @%s -> @%s\n",
ifb->name, thenb->name, elseb->name,
joinb->name);
convert(ifb, thenb, elseb, joinb);
}
if (debug['K']) {
fprintf(stderr, "\n> After if-conversion:\n");
printfn(fn, stderr);
}
}

236
loopopt.c Normal file
View File

@ -0,0 +1,236 @@
#include "all.h"
static int
forloop(Blk *bh, Blk *bb)
{
if (bh->npred == 2)
if (bh->jmp.type == Jjnz)
if (bb->npred == 1)
if (bb->jmp.type == Jjmp)
if (bb->jmp.s[0].b == bh)
return 1;
return 0;
}
static int
doloop(Blk *bh)
{
if (bh->npred == 2)
if (bh->jmp.type == Jjnz)
if (bh->jmp.s[0].b == bh || bh->jmp.s[1].b == bh)
return 1;
return 0;
}
static int
loopvar(Fn *fn, Blk *bh, Blk *bb, uint np, Ref *prl, Ref *pr0, Ref *pri)
{
Blk *bp;
Phi *p;
Suc *sb, *sp;
Tmp *t;
Ins *i;
assert(bh->npred == 2);
assert(bh->pred[0] == bb || bh->pred[1] == bb);
bp = bh->pred[bh->pred[0] == bb]; /* pre-header */
assert(np < bh->nphi);
p = &bh->phi[np];
if (KBASE(p->cls) != 0)
return 0; /* not integer */
sp = getsuc(bp, bh);
assert(sp->nr == bh->nphi);
*pr0 = sp->r[np];
sb = getsuc(bb, bh);
assert(sb->nr == bh->nphi);
*prl = sb->r[np];
if (rtype(*prl) != RTmp)
return 0; /* loop var must be an incremented tmp */
t = &fn->tmp[prl->val];
if (t->bid != bb->id || t->def == 0)
return 0; /* loop var must be defined by an add ins in the body */
i = t->def;
if (i->op != Oadd || req(i->arg[0], p->to) == req(i->arg[1], p->to))
return 0;
*pri = i->arg[req(i->arg[0], p->to)];
assert(p->cls == i->cls);
if (rtype(*pri) == RTmp) {
t = &fn->tmp[pri->val];
if (t->bid == bh->id || t->bid == bb->id)
return 0;
return 1;
}
else {
assert(rtype(*pri) == RCon);
return 1;
}
return 0;
}
static void
mulredux(Fn *fn, Blk *bh, Blk *bb, Blk *bp, uint np, Ref rl, Ref r0, Ref ri, Ins **pvins, uint *pnins)
{
Ins *i;
Tmp *t;
Ref r1, rlop;
Phi *p;
Suc *sb, *sp;
/* dead simple case for now... majority case in practice */
if (!req(r0, con01[0]) || !req(ri, con01[1]))
return;
assert(!req(rl, R)); /* rl unused */
assert(np < bh->nphi);
p = &bh->phi[np];
for (i = bb->ins; i < &bb->ins[bb->nins]; i++) {
/* TODO i->cls != p->cls is not necessary? */
if (i->op != Omul || i->cls != p->cls
|| req(i->arg[0], p->to) == req(i->arg[1], p->to))
continue;
r1 = i->arg[req(i->arg[0], p->to)];
if (rtype(r1) == RTmp) {
t = &fn->tmp[r1.val];
if (t->bid == bh->id || t->bid == bb->id)
continue;
} else
assert(rtype(r1) == RCon);
assert(KBASE(i->cls) == 0);
rlop = newtmp("lop", i->cls, fn);
vgrow(&bh->phi, ++bh->nphi);
bh->phi[bh->nphi-1] = (Phi){.to = i->to, .cls = i->cls};
sp = getsuc(bp, bh);
vgrow(&sp->r, ++sp->nr);
sp->r[sp->nr-1] = con01[0];
assert(sp->nr == bh->nphi);
sb = getsuc(bb, bh);
vgrow(&sb->r, ++sb->nr);
sb->r[sp->nr-1] = rlop;
assert(sb->nr == bh->nphi);
addins(pvins, pnins, &(Ins){.to = rlop, .op = Oadd, .cls = i->cls,
.arg = {i->to, r1}});
*i = (Ins){.op = Onop};
}
}
static void
addbase(Fn *fn, Blk *bh, Blk *bb, Blk *bp, uint np, Ref rl, Ref r0, Ref ri, Ins **pvins, uint *pnins)
{
Ins *i, *ii, *ib;
Tmp *t, *ti, *tb;
Use *u;
Ref rb;
Phi *p;
Suc *sp;
/* dead simple case for now... majority case in practice */
if (!req(r0, con01[0]))
return;
assert(!req(ri, R)); /* unused */
assert(pvins && pnins); /* unused */
assert(np < bh->nphi);
p = &bh->phi[np];
assert(rtype(p->to) == RTmp);
t = &fn->tmp[p->to.val];
if (t->nuse != 2)
return;
i = ib = 0;
for (u = t->use; u < &t->use[t->nuse]; u++) {
if (u->type != UIns)
return;
i = u->u.ins;
if (i->op != Oadd || i->cls != p->cls || u->bid != bb->id)
return;
assert(req(i->arg[0], p->to) || req(i->arg[1], p->to));
if (req(i->to, rl)) {
assert(rtype(i->to) == RTmp);
ti = &fn->tmp[i->to.val];
if (ti->nuse != 1)
return;
assert(ti->use[0].type == UPhi);
assert(ti->use[0].bid == bh->id);
assert(ti->use[0].u.p.np == np);
assert(ti->use[0].u.p.pbid == bb->id);
ii = i;
continue;
}
rb = i->arg[req(i->arg[0], p->to)];
if (req(rb, p->to))
return;
if (rtype(rb) == RTmp) {
tb = &fn->tmp[rb.val];
if (tb->bid == bh->id || tb->bid == bb->id)
return;
} else
assert(rtype(rb) == RCon);
ib = i;
}
assert(ii);
assert(ib);
sp = getsuc(bp, bh);
assert(sp->nr == bh->nphi);
assert(req(sp->r[np], con01[0]));
sp->r[np] = rb;
assert(req(ii->arg[0], p->to) != req(ii->arg[1], p->to));
ii->arg[!req(ii->arg[0], p->to)] = ib->to;
p->to = ib->to;
*ib = (Ins){.op = Onop};
}
typedef void optfn_t(Fn *, Blk *, Blk *, Blk *, uint, Ref, Ref, Ref, Ins **, uint *);
static void
loopvaropt(Fn *fn, optfn_t *optfn)
{
uint bid;
Blk *bh, *bb, *bp; /* header, body, pre-header */
Phi *p;
Ins *vins;
uint nins, np;
Ref rl, r0, ri; /* loop var, loop var init, loop var inc */
nins = 0;
vins = vnew(nins, sizeof vins[0], PFn); /* TODO use insb */
for (bid = 0; bid < fn->nblk; bid++) {
bh = fn->rpo[bid];
if (forloop(bh, bh->jmp.s[0].b))
bb = bh->jmp.s[0].b;
else if (forloop(bh, bh->jmp.s[1].b))
bb = bh->jmp.s[1].b;
else if (doloop(bh))
bb = bh; /* header and body */
else
continue;
bp = bh->pred[bh->pred[0] == bh];
assert(bh->loop > 1);
assert(bh->loop == bb->loop);
nins = 0;
for (np = 0; np < bh->nphi; np++)
if (loopvar(fn, bh, bb, np, &rl, &r0, &ri)) {
p = &bh->phi[np];
assert(KBASE(p->cls) == 0);
optfn(fn, bh, bb, bp, np, rl, r0, ri, &vins, &nins);
}
addnins(&bb->ins, &bb->nins, vins, nins);
}
}
void
loopopt(Fn *fn)
{
/* encourage simple loops */
ifelim(fn);
fillcfg(fn);
blkmerge(fn);
fillcfg(fn);
filluse(fn);
fillloop(fn);
loopvaropt(fn, mulredux);
filluse(fn);
loopvaropt(fn, addbase);
}

15
main.c
View File

@ -10,7 +10,8 @@ char debug['Z'+1] = {
['M'] = 0, /* memory optimization */ ['M'] = 0, /* memory optimization */
['N'] = 0, /* ssa construction */ ['N'] = 0, /* ssa construction */
['C'] = 0, /* copy elimination */ ['C'] = 0, /* copy elimination */
['F'] = 0, /* constant folding */ ['G'] = 0, /* gvn/gcm */
['K'] = 0, /* if-conversion */
['A'] = 0, /* abi lowering */ ['A'] = 0, /* abi lowering */
['I'] = 0, /* instruction selection */ ['I'] = 0, /* instruction selection */
['L'] = 0, /* liveness */ ['L'] = 0, /* liveness */
@ -20,6 +21,7 @@ char debug['Z'+1] = {
extern Target T_amd64_sysv; extern Target T_amd64_sysv;
extern Target T_amd64_apple; extern Target T_amd64_apple;
extern Target T_amd64_win;
extern Target T_arm64; extern Target T_arm64;
extern Target T_arm64_apple; extern Target T_arm64_apple;
extern Target T_rv64; extern Target T_rv64;
@ -27,6 +29,7 @@ extern Target T_rv64;
static Target *tlist[] = { static Target *tlist[] = {
&T_amd64_sysv, &T_amd64_sysv,
&T_amd64_apple, &T_amd64_apple,
&T_amd64_win,
&T_arm64, &T_arm64,
&T_arm64_apple, &T_arm64_apple,
&T_rv64, &T_rv64,
@ -76,11 +79,21 @@ func(Fn *fn)
ssacheck(fn); ssacheck(fn);
gvn(fn); gvn(fn);
fillcfg(fn); fillcfg(fn);
simplcfg(fn);
filluse(fn); filluse(fn);
filldom(fn); filldom(fn);
gcm(fn); gcm(fn);
filluse(fn); filluse(fn);
ssacheck(fn); ssacheck(fn);
loopopt(fn);
filluse(fn);
if (T.cansel) {
ifconvert(fn);
fillcfg(fn);
filluse(fn);
filldom(fn);
ssacheck(fn);
}
T.abi1(fn); T.abi1(fn);
simpl(fn); simpl(fn);
fillcfg(fn); fillcfg(fn);

22
ops.h
View File

@ -145,6 +145,8 @@ O(nop, T(x,x,x,x, x,x,x,x), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0)
O(addr, T(m,m,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0) O(addr, T(m,m,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0)
O(blit0, T(m,e,e,e, m,e,e,e), F(0,0,0,0,0,0,0,0,0,1)) X(0,1,0) V(0) O(blit0, T(m,e,e,e, m,e,e,e), F(0,0,0,0,0,0,0,0,0,1)) X(0,1,0) V(0)
O(blit1, T(w,e,e,e, x,e,e,e), F(0,0,0,0,0,0,0,0,0,1)) X(0,1,0) V(0) O(blit1, T(w,e,e,e, x,e,e,e), F(0,0,0,0,0,0,0,0,0,1)) X(0,1,0) V(0)
O(sel0, T(w,e,e,e, x,e,e,e), F(0,0,0,0,0,0,0,0,0,1)) X(0,0,0) V(0)
O(sel1, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,1)) X(0,0,0) V(0)
O(swap, T(w,l,s,d, w,l,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(1,0,0) V(0) O(swap, T(w,l,s,d, w,l,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(1,0,0) V(0)
O(sign, T(w,l,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0) O(sign, T(w,l,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(salloc, T(e,l,e,e, e,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0) O(salloc, T(e,l,e,e, e,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
@ -196,6 +198,26 @@ O(flagfne, T(x,x,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0)
O(flagfo, T(x,x,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0) O(flagfo, T(x,x,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0)
O(flagfuo, T(x,x,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0) O(flagfuo, T(x,x,e,e, x,x,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,1) V(0)
/* Backend Flag Select (Condition Move) */
O(xselieq, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xseline, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselisge, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselisgt, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselisle, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselislt, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xseliuge, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xseliugt, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xseliule, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xseliult, T(w,l,e,e, w,l,e,e), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfeq, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfge, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfgt, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfle, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselflt, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfne, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfo, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
O(xselfuo, T(e,e,s,d, e,e,s,d), F(0,0,0,0,0,0,0,0,0,0)) X(0,0,0) V(0)
#undef T #undef T
#undef X #undef X
#undef V #undef V

33
parse.c
View File

@ -57,6 +57,7 @@ enum Token {
Thlt, Thlt,
Texport, Texport,
Tthread, Tthread,
Textern,
Tcommon, Tcommon,
Tfunc, Tfunc,
Ttype, Ttype,
@ -116,6 +117,7 @@ static char *kwmap[Ntok] = {
[Thlt] = "hlt", [Thlt] = "hlt",
[Texport] = "export", [Texport] = "export",
[Tthread] = "thread", [Tthread] = "thread",
[Textern] = "extern",
[Tcommon] = "common", [Tcommon] = "common",
[Tfunc] = "function", [Tfunc] = "function",
[Ttype] = "type", [Ttype] = "type",
@ -215,12 +217,15 @@ getint()
n = 0; n = 0;
c = fgetc(inf); c = fgetc(inf);
m = (c == '-'); m = (c == '-');
if (m) if (m) {
c = fgetc(inf); c = fgetc(inf);
if (!isdigit(c))
err("integer expected");
}
do { do {
n = 10*n + (c - '0'); n = 10*n + (c - '0');
c = fgetc(inf); c = fgetc(inf);
} while ('0' <= c && c <= '9'); } while (isdigit(c));
ungetc(c, inf); ungetc(c, inf);
if (m) if (m)
n = 1 + ~n; n = 1 + ~n;
@ -424,11 +429,10 @@ static Ref
parseref() parseref()
{ {
Con c; Con c;
int tok;
memset(&c, 0, sizeof c); memset(&c, 0, sizeof c);
switch (next()) { switch ((tok = next())) {
default:
return R;
case Ttmp: case Ttmp:
return tmpref(tokval.str); return tmpref(tokval.str);
case Tint: case Tint:
@ -445,9 +449,20 @@ parseref()
c.bits.d = tokval.fltd; c.bits.d = tokval.fltd;
c.flt = 2; c.flt = 2;
break; break;
default:
for (;; tok=next()) {
switch (tok) {
case Textern:
c.sym.type |= SExt;
continue;
case Tthread: case Tthread:
c.sym.type = SThr; c.sym.type |= SThr;
expect(Tglo); continue;
}
break;
}
if (tok != Tglo)
return R;
/* fall through */ /* fall through */
case Tglo: case Tglo:
c.type = CAddr; c.type = CAddr;
@ -1252,7 +1267,9 @@ printcon(Con *c, FILE *f)
case CUndef: case CUndef:
break; break;
case CAddr: case CAddr:
if (c->sym.type == SThr) if (c->sym.type & SExt)
fprintf(f, "extern ");
if (c->sym.type & SThr)
fprintf(f, "thread "); fprintf(f, "thread ");
fprintf(f, "$%s", str(c->sym.id)); fprintf(f, "$%s", str(c->sym.id));
if (c->bits.i) if (c->bits.i)

View File

@ -131,7 +131,7 @@ slot(Ref r, Fn *fn)
static void static void
emitaddr(Con *c, FILE *f) emitaddr(Con *c, FILE *f)
{ {
assert(c->sym.type == SGlo); assert((c->sym.type & ~SExt) == SGlo);
fputs(str(c->sym.id), f); fputs(str(c->sym.id), f);
if (c->bits.i) if (c->bits.i)
fprintf(f, "+%"PRIi64, c->bits.i); fprintf(f, "+%"PRIi64, c->bits.i);
@ -231,7 +231,20 @@ loadaddr(Con *c, char *rn, FILE *f)
{ {
char off[32]; char off[32];
if (c->sym.type == SThr) { switch (c->sym.type) {
case SGlo:
fprintf(f, "\tlui %s, %%hi(", rn);
emitaddr(c, f);
fprintf(f, ")\n\taddi %s, %s, %%lo(", rn, rn);
emitaddr(c, f);
fputs(")\n", f);
break;
case SExt:
fprintf(f, "\tla %s, ", rn);
emitaddr(c, f);
fputc('\n', f);
break;
case SThr:
if (c->bits.i) if (c->bits.i)
sprintf(off, "+%"PRIi64, c->bits.i); sprintf(off, "+%"PRIi64, c->bits.i);
else else
@ -242,10 +255,9 @@ loadaddr(Con *c, char *rn, FILE *f)
rn, rn, str(c->sym.id), off); rn, rn, str(c->sym.id), off);
fprintf(f, "\taddi %s, %s, %%tprel_lo(%s)%s\n", fprintf(f, "\taddi %s, %s, %%tprel_lo(%s)%s\n",
rn, rn, str(c->sym.id), off); rn, rn, str(c->sym.id), off);
} else { break;
fprintf(f, "\tla %s, ", rn); case SExtThr:
emitaddr(c, f); die("extern thread unavailable on rv64");
fputc('\n', f);
} }
} }
@ -282,7 +294,7 @@ fixmem(Ref *pr, Fn *fn, FILE *f)
if (rtype(r) == RCon) { if (rtype(r) == RCon) {
c = &fn->con[r.val]; c = &fn->con[r.val];
if (c->type == CAddr) if (c->type == CAddr)
if (c->sym.type == SThr) { if (c->sym.type != SGlo) {
loadcon(c, T6, Kl, f); loadcon(c, T6, Kl, f);
*pr = TMP(T6); *pr = TMP(T6);
} }
@ -387,7 +399,7 @@ emitins(Ins *i, Fn *fn, FILE *f)
case RCon: case RCon:
con = &fn->con[i->arg[0].val]; con = &fn->con[i->arg[0].val];
if (con->type != CAddr if (con->type != CAddr
|| con->sym.type != SGlo || (con->sym.type & SThr)
|| con->bits.i) || con->bits.i)
goto Invalid; goto Invalid;
fprintf(f, "\tcall %s\n", str(con->sym.id)); fprintf(f, "\tcall %s\n", str(con->sym.id));
@ -444,7 +456,7 @@ rv64_emitfn(Fn *fn, FILE *f)
static int id0; static int id0;
int lbl, neg, off, frame, *pr, r; int lbl, neg, off, frame, *pr, r;
Blk *b, *s; Blk *b, *s;
Ins *i; Ins *i, ii;
emitfnlnk(fn->name, &fn->lnk, f); emitfnlnk(fn->name, &fn->lnk, f);
@ -549,6 +561,11 @@ rv64_emitfn(Fn *fn, FILE *f)
b->s2 = s; b->s2 = s;
neg = 1; neg = 1;
} }
if (rtype(b->jmp.arg) == RSlot) {
ii.arg[0] = b->jmp.arg;
emitf("lw t6, %M0", &ii, fn, f);
b->jmp.arg = TMP(T6);
}
assert(isreg(b->jmp.arg)); assert(isreg(b->jmp.arg));
fprintf(f, fprintf(f,
"\tb%sz %s, .L%d\n", "\tb%sz %s, .L%d\n",

View File

@ -31,6 +31,7 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
c = &fn->con[r0.val]; c = &fn->con[r0.val];
if (c->type == CAddr && memarg(r, op, i)) if (c->type == CAddr && memarg(r, op, i))
break; break;
if (KBASE(k) == 0)
if (c->type == CBits && immarg(r, op, i)) if (c->type == CBits && immarg(r, op, i))
if (-2048 <= c->bits.i && c->bits.i < 2048) if (-2048 <= c->bits.i && c->bits.i < 2048)
break; break;
@ -41,7 +42,7 @@ fixarg(Ref *r, int k, Ins *i, Fn *fn)
* immediates * immediates
*/ */
assert(c->type == CBits); assert(c->type == CBits);
n = stashbits(&c->bits, KWIDE(k) ? 8 : 4); n = stashbits(c->bits.i, KWIDE(k) ? 8 : 4);
vgrow(&fn->con, ++fn->ncon); vgrow(&fn->con, ++fn->ncon);
c = &fn->con[fn->ncon-1]; c = &fn->con[fn->ncon-1];
sprintf(buf, "\"%sfp%d\"", T.asloc, n); sprintf(buf, "\"%sfp%d\"", T.asloc, n);

View File

@ -50,6 +50,7 @@ Target T_rv64 = {
.emitfn = rv64_emitfn, .emitfn = rv64_emitfn,
.emitfin = elf_emitfin, .emitfin = elf_emitfin,
.asloc = ".L", .asloc = ".L",
.cansel = 0,
}; };
MAKESURE(rsave_size_ok, sizeof rv64_rsave == (NGPS+NFPS+1) * sizeof(int)); MAKESURE(rsave_size_ok, sizeof rv64_rsave == (NGPS+NFPS+1) * sizeof(int));

22
spill.c
View File

@ -406,26 +406,20 @@ spill(Fn *fn)
if (rtype(b->jmp.arg) == RCall) if (rtype(b->jmp.arg) == RCall)
v->t[0] |= T.retregs(b->jmp.arg, 0); v->t[0] |= T.retregs(b->jmp.arg, 0);
} }
if (rtype(b->jmp.arg) == RTmp) {
t = b->jmp.arg.val;
assert(KBASE(tmp[t].cls) == 0);
bsset(v, t);
limit2(v, 0, 0, NULL);
if (!bshas(v, t))
b->jmp.arg = slot(t);
}
for (t=Tmp0; bsiter(b->out, &t); t++) for (t=Tmp0; bsiter(b->out, &t); t++)
if (!bshas(v, t)) if (!bshas(v, t))
slot(t); slot(t);
bscopy(b->out, v); bscopy(b->out, v);
/* 2. process the block instructions */ /* 2. process the block instructions */
if (rtype(b->jmp.arg) == RTmp) {
t = b->jmp.arg.val;
assert(KBASE(tmp[t].cls) == 0);
lvarg[0] = bshas(v, t);
bsset(v, t);
bscopy(u, v);
limit2(v, 0, 0, NULL);
if (!bshas(v, t)) {
if (!lvarg[0])
bsclr(u, t);
b->jmp.arg = slot(t);
}
reloads(u, v);
}
curi = &insb[NIns]; curi = &insb[NIns];
for (i=&b->ins[b->nins]; i!=b->ins;) { for (i=&b->ins[b->nins]; i!=b->ins;) {
i--; i--;

View File

@ -28,7 +28,7 @@ function $test() {
# >>> driver # >>> driver
# #include <stdio.h> # #include <stdio.h>
# struct four { long l; char c; int i; }; # struct four { long long l; char c; int i; };
# extern void test(void); # extern void test(void);
# int F(int a0, int a1, int a2, int a3, struct four s, int a6) { # int F(int a0, int a1, int a2, int a3, struct four s, int a6) {
# printf("%d %d %d %d %d %d %d\n", # printf("%d %d %d %d %d %d %d\n",

View File

@ -107,7 +107,7 @@ function $test() {
# typedef struct { int i; } st2; # typedef struct { int i; } st2;
# typedef struct { float f; int i; } st3; # typedef struct { float f; int i; } st3;
# typedef struct { int i; double d; } st4; # typedef struct { int i; double d; } st4;
# typedef struct { float f; long l; } st5; # typedef struct { float f; long long l; } st5;
# typedef struct { char t[16]; } st6; # typedef struct { char t[16]; } st6;
# typedef struct { float f; double d; } st7; # typedef struct { float f; double d; } st7;
# typedef struct { int i[4]; } st8; # typedef struct { int i[4]; } st8;

View File

@ -150,7 +150,7 @@ function w $main() {
# typedef struct { float s0, s1; } Sss; # typedef struct { float s0, s1; } Sss;
# typedef struct { float s; double d; } Ssd; # typedef struct { float s; double d; } Ssd;
# typedef struct { int w0, w1; } Sww; # typedef struct { int w0, w1; } Sww;
# typedef struct { long l; char b; } Slb; # typedef struct { long long l; char b; } Slb;
# typedef struct { char b[17]; } Sbig; # typedef struct { char b[17]; } Sbig;
# typedef struct { double d0, d1, d2; } Sddd; # typedef struct { double d0, d1, d2; } Sddd;
# Sfi1 zfi1, fi1 = { -123, 4.56 }; # Sfi1 zfi1, fi1 = { -123, 4.56 };
@ -168,11 +168,11 @@ function w $main() {
# void pss(Sss *s) { printf(" { %g, %g }", s->s0, s->s1); } # void pss(Sss *s) { printf(" { %g, %g }", s->s0, s->s1); }
# void psd(Ssd *s) { printf(" { %g, %g }", s->s, s->d); } # void psd(Ssd *s) { printf(" { %g, %g }", s->s, s->d); }
# void pww(Sww *s) { printf(" { %d, %d }", s->w0, s->w1); } # void pww(Sww *s) { printf(" { %d, %d }", s->w0, s->w1); }
# void plb(Slb *s) { printf(" { %ld, '%c' }", s->l, s->b); } # void plb(Slb *s) { printf(" { %lld, '%c' }", s->l, s->b); }
# void pbig(Sbig *s) { printf(" \"%.17s\"", s->b); } # void pbig(Sbig *s) { printf(" \"%.17s\"", s->b); }
# void pddd(Sddd *s) { printf(" { %g, %g, %g }", s->d0, s->d1, s->d2); } # void pddd(Sddd *s) { printf(" { %g, %g, %g }", s->d0, s->d1, s->d2); }
# void pw(int w) { printf(" %d", w); } # void pw(int w) { printf(" %d", w); }
# void pl(long l) { printf(" %ld", l); } # void pl(long long l) { printf(" %lld", l); }
# void ps(float s) { printf(" %g", s); } # void ps(float s) { printf(" %g", s); }
# void pd(double d) { printf(" %g", d); } # void pd(double d) { printf(" %g", d); }
# /* --------------------------- */ # /* --------------------------- */
@ -206,8 +206,8 @@ function w $main() {
# pss(&p0); puts(""); # pss(&p0); puts("");
# qfn4(p0); # qfn4(p0);
# } # }
# extern void qfn5(double, double, double, double, double, double, double, Sss, float, long); # extern void qfn5(double, double, double, double, double, double, double, Sss, float, long long);
# void cfn5(double p0, double p1, double p2, double p3, double p4, double p5, double p6, Sss p7, float p8, long p9) { # void cfn5(double p0, double p1, double p2, double p3, double p4, double p5, double p6, Sss p7, float p8, long long p9) {
# printf("qbe->c(%d)", 5); # printf("qbe->c(%d)", 5);
# pss(&p7); ps(p8); pl(p9); puts(""); # pss(&p7); ps(p8); pl(p9); puts("");
# qfn5(p0, p1, p2, p3, p4, p5, p6, p7, p8, p9); # qfn5(p0, p1, p2, p3, p4, p5, p6, p7, p8, p9);
@ -236,8 +236,8 @@ function w $main() {
# pbig(&p0); puts(""); # pbig(&p0); puts("");
# qfn9(p0); # qfn9(p0);
# } # }
# extern void qfn10(int, int, int, int, int, int, int, int, Sbig, float, long); # extern void qfn10(int, int, int, int, int, int, int, int, Sbig, float, long long);
# void cfn10(int p0, int p1, int p2, int p3, int p4, int p5, int p6, int p7, Sbig p8, float p9, long p10) { # void cfn10(int p0, int p1, int p2, int p3, int p4, int p5, int p6, int p7, Sbig p8, float p9, long long p10) {
# printf("qbe->c(%d)", 10); # printf("qbe->c(%d)", 10);
# pbig(&p8); ps(p9); pl(p10); puts(""); # pbig(&p8); ps(p9); pl(p10); puts("");
# qfn10(p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10); # qfn10(p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10);

20
test/abi9.ssa Normal file
View File

@ -0,0 +1,20 @@
type :obj = { l, l, l, l }
export
function :obj $f(l %self) {
@_0
%_1 =l alloc8 16
storel 77, %_1
ret %_1
}
# >>> driver
# #include <stdio.h>
# typedef struct { long long a, b, c, d; } obj;
# extern obj f();
# int main() { obj ret = f(); printf("%lld\n", ret.a); return 0; }
# <<<
# >>> output
# 77
# <<<

View File

@ -1,3 +1,4 @@
# skip amd64_win (no signals on win32)
# test amd64 addressing modes # test amd64 addressing modes
export export

View File

@ -1,4 +1,4 @@
# skip arm64 arm64_apple rv64 # skip arm64 arm64_apple rv64 amd64_win
# a hack example, # a hack example,
# we use a dark type to get # we use a dark type to get
# a pointer to the stack. # a pointer to the stack.

238
test/ifc.ssa Normal file
View File

@ -0,0 +1,238 @@
export
function l $ifc1(l %v0, l %v1, w %c) {
@start
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifc2(l %v0, l %v1, w %p) {
@start
%c =w cnew %p, 42
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifc3(l %v0, l %v1, w %p) {
@start
%c =w cugtw %p, 42
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifclts(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w clts %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifcles(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w cles %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifcgts(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w cgts %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifcges(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w cges %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifceqs(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w ceqs %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifcnes(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w cnes %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifcos(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w cos %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
export
function l $ifcuos(s %s0, s %s1, l %v0, l %v1) {
@start
%c =w cuos %s0, %s1
jnz %c, @true, @false
@true
%v =l copy %v1
jmp @end
@false
%v =l copy %v0
jmp @end
@end
ret %v
}
# >>> driver
# extern long ifc1(long, long, int);
# extern long ifc2(long, long, int);
# extern long ifc3(long, long, int);
# extern long ifclts(float, float, long, long);
# extern long ifcles(float, float, long, long);
# extern long ifcgts(float, float, long, long);
# extern long ifcges(float, float, long, long);
# extern long ifceqs(float, float, long, long);
# extern long ifcnes(float, float, long, long);
# extern long ifcos(float, float, long, long);
# extern long ifcuos(float, float, long, long);
# int main() {
# return
# ifc1(7, 5, 0) != 7
# || ifc1(7, 5, 1) != 5
# || ifc1(7, 5, 33) != 5
# || ifc2(7, 5, 42) != 7
# || ifc2(7, 5, 41) != 5
# || ifc2(7, 5, 43) != 5
# || ifc3(7, 5, 42) != 7
# || ifc3(7, 5, 41) != 7
# || ifc3(7, 5, 43) != 5
# || ifclts(5.0f, 6.0f, 7, 5) != 5
# || ifclts(5.0f, 5.0f, 7, 5) != 7
# || ifclts(5.0f, 4.0f, 7, 5) != 7
# || ifclts(5.0f, 0.0f/0.0f, 7, 5) != 7
# || ifclts(0.0f/0.0f, 5.0f, 7, 5) != 7
# || ifclts(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcles(5.0f, 6.0f, 7, 5) != 5
# || ifcles(5.0f, 5.0f, 7, 5) != 5
# || ifcles(5.0f, 4.0f, 7, 5) != 7
# || ifcles(5.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcles(0.0f/0.0f, 5.0f, 7, 5) != 7
# || ifcles(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcgts(5.0f, 6.0f, 7, 5) != 7
# || ifcgts(5.0f, 5.0f, 7, 5) != 7
# || ifcgts(5.0f, 4.0f, 7, 5) != 5
# || ifcgts(5.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcgts(0.0f/0.0f, 5.0f, 7, 5) != 7
# || ifcgts(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcges(5.0f, 6.0f, 7, 5) != 7
# || ifcges(5.0f, 5.0f, 7, 5) != 5
# || ifcges(5.0f, 4.0f, 7, 5) != 5
# || ifcges(5.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcges(0.0f/0.0f, 5.0f, 7, 5) != 7
# || ifcges(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 7
# || ifceqs(5.0f, 6.0f, 7, 5) != 7
# || ifceqs(5.0f, 5.0f, 7, 5) != 5
# || ifceqs(5.0f, 4.0f, 7, 5) != 7
# || ifceqs(5.0f, 0.0f/0.0f, 7, 5) != 7
# || ifceqs(0.0f/0.0f, 5.0f, 7, 5) != 7
# || ifceqs(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcnes(5.0f, 6.0f, 7, 5) != 5
# || ifcnes(5.0f, 5.0f, 7, 5) != 7
# || ifcnes(5.0f, 4.0f, 7, 5) != 5
# || ifcnes(5.0f, 0.0f/0.0f, 7, 5) != 5
# || ifcnes(0.0f/0.0f, 5.0f, 7, 5) != 5
# || ifcnes(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 5
# || ifcos(5.0f, 6.0f, 7, 5) != 5
# || ifcos(5.0f, 5.0f, 7, 5) != 5
# || ifcos(5.0f, 4.0f, 7, 5) != 5
# || ifcos(5.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcos(0.0f/0.0f, 5.0f, 7, 5) != 7
# || ifcos(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 7
# || ifcuos(5.0f, 6.0f, 7, 5) != 7
# || ifcuos(5.0f, 5.0f, 7, 5) != 7
# || ifcuos(5.0f, 4.0f, 7, 5) != 7
# || ifcuos(5.0f, 0.0f/0.0f, 7, 5) != 5
# || ifcuos(0.0f/0.0f, 5.0f, 7, 5) != 5
# || ifcuos(0.0f/0.0f, 0.0f/0.0f, 7, 5) != 5
# ;
# }
# <<<

38
test/isel6.ssa Normal file
View File

@ -0,0 +1,38 @@
# make sure large consts are lowered
# without an offset
# i.e. not movq $9223372036854775807, 64(%rax)
export function w $main() {
@_0
%_1 =w call $myfunc(l 1, l 2, l 3, l 4, l 5, l 6, l 7, l 8, l 9223372036854775807)
ret 0
}
# >>> driver
# #include <stdio.h>
# #include <stdint.h>
# #include <inttypes.h>
# void myfunc(int64_t a, int64_t b, int64_t c, int64_t d, int64_t e, int64_t f, int64_t g, int64_t h, int64_t i) {
# printf("%" PRId64 "\n", a);
# printf("%" PRId64 "\n", b);
# printf("%" PRId64 "\n", c);
# printf("%" PRId64 "\n", d);
# printf("%" PRId64 "\n", e);
# printf("%" PRId64 "\n", f);
# printf("%" PRId64 "\n", g);
# printf("%" PRId64 "\n", h);
# printf("%" PRId64 "\n", i);
# }
# <<<
# >>> output
# 1
# 2
# 3
# 4
# 5
# 6
# 7
# 8
# 9223372036854775807
# <<<

View File

@ -1,3 +1,4 @@
# skip amd64_win (pthread and tls not implemented)
thread data $i = align 4 {w 42} thread data $i = align 4 {w 42}
data $fmti = align 1 {b "i%d==%d\n", b 0} data $fmti = align 1 {b "i%d==%d\n", b 0}

View File

@ -29,7 +29,7 @@ char *tok[] = {
"function", "type", "data", "section", "align", "dbgfile", "function", "type", "data", "section", "align", "dbgfile",
"blit", "l", "w", "sh", "uh", "h", "sb", "ub", "b", "blit", "l", "w", "sh", "uh", "h", "sb", "ub", "b",
"d", "s", "z", "loadw", "loadl", "loads", "loadd", "d", "s", "z", "loadw", "loadl", "loads", "loadd",
"alloc1", "alloc2", "thread", "common", "alloc1", "alloc2", "thread", "extern", "common",
}; };
enum { enum {

View File

@ -31,17 +31,23 @@ find_cc_and_qemu() {
cc=$candidate_cc cc=$candidate_cc
echo "cc: $cc" echo "cc: $cc"
if [ "$target" = "$(uname -m)" ]; then if [ "$target" = "$(uname -m)" ]
then
qemu=qemu_not_needed qemu=qemu_not_needed
echo "qemu: not needed, testing native architecture" echo "qemu: not needed, testing native architecture"
else else
qemu="$3" qemu="$3"
if $qemu -version >/dev/null 2>&1; then if $qemu -version >/dev/null 2>&1
then
sysroot=$($candidate_cc -print-sysroot) sysroot=$($candidate_cc -print-sysroot)
if [ -n "$sysroot" ]; then if [ -n "$sysroot" ]; then
qemu="$qemu -L $sysroot" qemu="$qemu -L $sysroot"
fi fi
echo "qemu: $qemu" echo "qemu: $qemu"
elif $qemu --version >/dev/null 2>&1
then
# wine
:
else else
qemu= qemu=
echo "qemu: not found" echo "qemu: not found"
@ -90,6 +96,19 @@ init() {
fi fi
bin="$bin -t amd64_sysv" bin="$bin -t amd64_sysv"
;; ;;
amd64_win)
for p in x86_64-w64-mingw32
do
find_cc_and_qemu x86_64-w64 "$p-gcc -static" "wine"
done
if test -z "$cc"
then
echo "Cannot find windows compiler or wine."
exit 1
fi
export WINEDEBUG=-all
bin="$bin -t amd64_win"
;;
"") "")
case `uname` in case `uname` in
*Darwin*) *Darwin*)
@ -185,7 +204,7 @@ once() {
if test -s $out if test -s $out
then then
$qemu $exe a b c | diff -u - $out $qemu $exe a b c | tr -d '\r' | diff -u - $out
ret=$? ret=$?
reason="output" reason="output"
else else

40
util.c
View File

@ -164,7 +164,7 @@ addins(Ins **pvins, uint *pnins, Ins *i)
} }
void void
addbins(Blk *b, Ins **pvins, uint *pnins) addbins(Ins **pvins, uint *pnins, Blk *b)
{ {
Ins *i; Ins *i;
@ -281,6 +281,17 @@ igroup(Blk *b, Ins *i, Ins **i0, Ins **i1)
assert(i < ie); assert(i < ie);
*i1 = i + 1; *i1 = i + 1;
return; return;
case Osel1:
for (; i>ib && (i-1)->op == Osel1; i--)
;
assert(i->op == Osel0);
/* fall through */
case Osel0:
*i0 = i++;
for (; i<ie && i->op == Osel1; i++)
;
*i1 = i;
return;
default: default:
if (ispar(i->op)) if (ispar(i->op))
goto case_Opar; goto case_Opar;
@ -343,23 +354,16 @@ static int cmptab[][2] ={
[Cisge] = {Cislt, Cisle}, [Cisge] = {Cislt, Cisle},
[Cieq] = {Cine, Cieq}, [Cieq] = {Cine, Cieq},
[Cine] = {Cieq, Cine}, [Cine] = {Cieq, Cine},
[NCmpI+Cfle] = {NCmpI+Cfgt, NCmpI+Cfge}, [NCmpI+Cfle] = {-1, NCmpI+Cfge},
[NCmpI+Cflt] = {NCmpI+Cfge, NCmpI+Cfgt}, [NCmpI+Cflt] = {-1, NCmpI+Cfgt},
[NCmpI+Cfgt] = {NCmpI+Cfle, NCmpI+Cflt}, [NCmpI+Cfgt] = {-1, NCmpI+Cflt},
[NCmpI+Cfge] = {NCmpI+Cflt, NCmpI+Cfle}, [NCmpI+Cfge] = {-1, NCmpI+Cfle},
[NCmpI+Cfeq] = {NCmpI+Cfne, NCmpI+Cfeq}, [NCmpI+Cfeq] = {-1, NCmpI+Cfeq},
[NCmpI+Cfne] = {NCmpI+Cfeq, NCmpI+Cfne}, [NCmpI+Cfne] = {-1, NCmpI+Cfne},
[NCmpI+Cfo] = {NCmpI+Cfuo, NCmpI+Cfo}, [NCmpI+Cfo] = {-1, NCmpI+Cfo},
[NCmpI+Cfuo] = {NCmpI+Cfo, NCmpI+Cfuo}, [NCmpI+Cfuo] = {-1, NCmpI+Cfuo},
}; };
int
cmpneg(int c)
{
assert(0 <= c && c < NCmp);
return cmptab[c][0];
}
int int
cmpop(int c) cmpop(int c)
{ {
@ -371,9 +375,9 @@ int
cmpwlneg(int op) cmpwlneg(int op)
{ {
if (INRANGE(op, Ocmpw, Ocmpw1)) if (INRANGE(op, Ocmpw, Ocmpw1))
return cmpneg(op - Ocmpw) + Ocmpw; return cmptab[op - Ocmpw][0] + Ocmpw;
if (INRANGE(op, Ocmpl, Ocmpl1)) if (INRANGE(op, Ocmpl, Ocmpl1))
return cmpneg(op - Ocmpl) + Ocmpl; return cmptab[op - Ocmpl][0] + Ocmpl;
die("not a wl comparison"); die("not a wl comparison");
} }