Building packages on exotic archs

What we learnt from it, and why participating in global warming

Landry Breuil <landry@openbsd.org>

EuroBSDCon 2016 / Sep,25 / Belgrade

Who ?

Archs

13 as of 6.0
of those, 8 have packages
vax last had packages in 5.8, sh and sparc in 5.9
armv7 incoming !

Me ?

hppa since feb 2010 (after 4.7, previously simon@)
sparc64 since march 2010 (previously sturm@)
powerpc since july 2013 (after 5.4, previously ajacoutot@)
alpha since july 2013 (previously merdely@)
trend ?

Hardware

hppa: 2x J6700, 550MHz & 875MHz, MP
sparc64: v215 + 2 * v210, non-MP
powerpc: 2 Xserve G4, 1GHz, MP (previously 3, one coming ?)
alpha: CS20D 833 MHz, MP (previously 2, died a year ago)
8/10 hosts to babysit via remote serial console

Setup

dpb(1)
proot(1)
NFS via hub.p for distfiles/logs/packages/plist
NFS for the arch for /usr/ports
wrapper scripts...

Workflow

reboot on bsd.rd
sysmerge
mkproot.sh
cvs up
build.sh
... wait ...
report_build.sh
sign, upload

Results over time

Build times comparison

Depends where the jobs are dispatched (master/slave), if big packages build..

arch	Gtk+3	QT4	Full bulk
i386	12m	<3h	24h
amd64	20m	~3/4h	17h
hppa	1h45/2h	<24h	5/10d
sparc64	1h30	~19h	2/3w
macppc	2h	~26h	2/3w
alpha	<2h30	~35h	2/3w

Usefulness ?

for 3 archs, it takes two/three weeks to build and ship a package snapshots
there's a library version bump in base (libc, librthread, libssl, libcrypto..) at least every two/three weeks
oops.
but it is okay for releases too, right ?

Usefulness ?

August 2016 downloads on ftp.fr, snapshots and releases, not excluding dupes in IPs, excluding bots

arch	pkgs	quirks
amd64	250k/w	6k/w
i386	30k/w	700/800/w
macppc	5k/m	75/m
sparc64	300/m	7/m
hppa	0	0
alpha	0	0

	zegrep -c /packages/$arch/.*.tgz.*OpenBSD.ftp /var/www/logs/access.log*

Usefulness ?

Provides a complete (?) stresstest for the base OS
make release isn't
load + network + disk + memory + ...
Over the past 6 years, SMP was enabled on those archs... dogfooding anyone?
PIE, W^X, ...
Exercizes the whole toolchain (binutils, gcc...)

When the s*** hits the fan

Platform^W^WToolchain deficiencies

atomics: none on hppa/arm, no 64-bits on macppc (__sync_val_compare_and_swap_8)
powerpc: relocation overflows: relocation truncated to fit: R_PPC_PLTREL24 - too many symbols exported ?
powerpc: ld --gc-sections (fixed thanks to jca@)
segfaults in the toolchain, gcc ICEs

BROKEN-hppa= bend.c:155:12: internal compiler error: in expand_expr_addr_expr_1, at expr.c:7697

Bad coding practices

-Werror + ugly casts.. fail

speaking of casts.. graphics/blender:

(CUdeviceptr)((uchar*)mem.device_pointer + offset)
error: cast from 'ccl::uchar*' to 'CUdeviceptr' loses precision

Hw/OS assumptions

alpha ? you must be a digital unix/tru64 bearded expert
sgi ? probably running irix
hppa ? hpux business consultant
sparc ? you're a solaris zealot
macppc ? steve jobs fanboy

I'm not a number, i'm a free man!

#ifdef sparc
#include <malloc.h>
#endif

Mo-mo-monster code

Ginormous c++ triggering OOM: hello mapnik, boost, games, emulators
But random failures...
Ruby on hppa/powerpc: OOM when building ext/ripper/ripper.c
Mozilla linking libxul.so on 32 bits: ld --no-keep-memory
And lots others..

all the world is x86 (or arm!)

freerdp on sparc64 (fixed)

cc1: error: unrecognized command line option "-mfpu=neon" "-mfloat-abi=softfp"

#error strikes all around

#error PostgreSQL does not have native spinlock support on this platform.
#error Please implement for your platform (sdl2)
#error Libzrtp can't detect byte order
#error unsupported platform (moarvm)
#error "no memory barrier implementation" (rubinius)

Set ONLY_FOR_ARCHS ?
Or proper feature detection
cpuid randr runtime detection in rspamd on macppc (fixed)

error: unknown register name 'edx' in 'asm'

all the world is big endian!

Little-endian codepaths probably never tested..

..
#elif defined(__sparc) || defined(__sparc__) || \
      defined(_POWER) || defined(__hppa) || \
      defined(_MIPSEB) || defined(_BIG_ENDIAN)
..

blindly add ||defined(arch)? or just use #include <endian.h> FFS!

But should i use BYTE_ORDER, _BYTE_ORDER, __BYTE_ORDER, or __BYTE_ORDER__ ? __ORDER_PDP_ENDIAN__ ?

Sloppy port workarounds

MODULES +=              gcc4 lang/clang
MODGCC4_ARCHS =         powerpc sparc64 alpha
MODGCC4_LANGS =         c c++
MODCLANG_ARCHS =        amd64 i386
MODCLANG_LANGS =        c c++

GCC 4.6 on alpha, 4.9 on others.... but what about hppa here ?

Real problems in OpenBSD (headers)

Fixes in the base system/kernel

(re-)Fixes at least net/net-snmp, sysutils/gkrellm/gkrellm,-client, sysutils/xuvmstat and sysutils/bubblemon-dockapp on alpha.

But you need to get a basesys/kernel developer involved...

The bad & ugly

Lots of code builds, but is never run... except when running tests, or validating something during fake

We keep stuff building/packaging just for dependencies even if they're known broken (hello webkit, spidermonkey...)

Long way to go, but..

We have the BROKEN marker

sqlite> select arch, count(fullpkgpath) as c from broken group by arch order by c desc;
hppa|197
sparc64|58
alpha|57
...
macppc|4

http://build-failures.rhaalovely.net/

Hopeless ?

If you use something, show some love and fixit

http://v4.freshbsd.org/search?q=BROKEN&committer[]=landry

src

Questions ?

Slides on http://rhaalovely.net/~landry/eurobsdcon2016/

Building packages on exotic archs

What we learnt from it, and why participating in global warming

Landry Breuil <landry@openbsd.org>

EuroBSDCon 2016 / Sep,25 / Belgrade

Who ?

Archs

Me ?

Hardware

Setup

Workflow

Results over time

Build times comparison

Usefulness ?

Usefulness ?

Usefulness ?

When the s*** hits the fan

Platform^W^WToolchain deficiencies

Bad coding practices

Hw/OS assumptions

Mo-mo-monster code

all the world is x86 (or arm!)

all the world is big endian!

Sloppy port workarounds

Real problems in OpenBSD (headers)

The bad & ugly

Long way to go, but..

Hopeless ?

Questions ?

Thanks for listening, thanks to EuroBSDCon for the trip!