Mozilla on OpenBSD

When the lizard meets puffy

Landry Breuil <landry@openbsd.org>

BSDCan 2013 / May,17 / Ottawa

Who ?

OpenBSD

As a project

• CVS (every time you use it, a kitten dies)
• 6-month release schedule, 1-year support
• Evolution, not revolution
• Hostility towards gplv3 (ie old toolchain in base)
• "Shut up and hack" & "eat your own dogfood"
• ~120 active developers, ~30/40 'porters'
• Regular hackathons all around the world

As a desktop/laptop OS

• use -CURRENT
• binary packages / pkg_add -u since 2005
• Xfce 4.10, GNOME 3.8, KDE 4.10 (wip)
• Chromium, LibreOffice, Mplayer, games...
• suspend/resume via our own ACPI implem
• DRM on intel/ati, KMS on intel
• rthreads!
• no udev, maybe someday?

Mozilla, the project

Not only firefox

• Gecko, the engine (xulrunner)
• Thunderbird, still rocking w/ lightning + sogo extension
• Seamonkey, the venerable Mozilla suite
• Instantbird, the IM client
• Firefox Sync
• Addons
• Mozilla Persona
• B2G

B2G/firefox OS

The web is the platform

• Gecko on top of Linux kernel (Gonk) + HAL + Android libs
• Gaia : default set of apps / userland
• HTML/CSS interface + JS engine
• WebAPIs (audio/video/sms/battery/vibrate/phone/...)
• Installable webapps (also installable to desktop!)
• No central closed marketplace
• Geeksphone shipping to developers (sold out)

Fast release schedule

Platforms

• Tier1
• Android/arm (gcc)
• Linux/x86(-64) (gcc) (in // with distrib maintainers)
• OSX/x86(-64) (clang)
• Windows/x86 (msvc)
• Tier2/Tier3.. the others (os/2 or aix, anyone ?)
• Crazy people like TenFourFox, Classilla or Bezilla
• And us!

Community

• Thousands of developers/translators
• But also.. QA/Support, Bug Triage, IT, UX, Marketing
• WebFWD / mozilla spaces : startup incubator
• WebMaker: let users become creators
• MoCo is hiring: http://careers.mozilla.org/

Communication

• irc://irc.mozilla.org/ (#developers, #maildev, #jsapi, #gfx..)
• http://planet.mozilla.org
• Newsgroups^WGooglegroups

Mozilla, the tools

Mercurial

Queues

hg qpop, hg qseries, hg qnew, hg qpush, hg qfinish...

Branches / inbound

Sheriffs watch the tree, star failures, and handle merges

Project repositories (ionmonkey, build-system, birch, alder..)

Bugzilla a.k.a. bmo

• Everything is handled there (code, hw, accounts, releng..)
• No commit without a bug #
• Splinter for code review
• Huge use of depends/blocks chains
• User / Product dashboards
• Integration with MQ : bzexport & qimportbz
• Integration with MUA : bzhelper & REST BzAPI

Workflow

• Bug
• Patch against m-c
• r?
• repeat previous until r+
• checkin-needed / hg push
• (watch the tree)
• approval-{aurora,beta,release}? (optional)

Tbpl (a.k.a. TinderboxPushLog)

http://tbpl.mozilla.org

Try

dawn:~/src/mozilla-central/ $hg qseries
bug-844818-separate-alsa-pulse-webrtc
bug-844430-libsctp-openbsd
bug-807492-webrtc-bsd
bug-807492-webrtc-openbsd-fixes
try
dawn:~/src/mozilla-central/ $hg log -l1 | grep summary
summary:     try: -b do -p all -u all -t all
dawn:~/src/mozilla-central/ $hg push -f ssh://hg.mozilla.org/try
	

Other web tools

• HG : http://hg.mozilla.org/
• MXR : http://mxr.mozilla.org/ (30Gb of code indexed..)
• socorro : https://crash-stats.mozilla.com/
• AMO : http://addons.mozilla.org/
• MDN, WMO, just a galaxy of websites...

Commit bit

• File a bug!
• Level 1: access to try, need 1 voucher
• SSH key / ldap account / committer's agreement
• Keep nagging ppl with commit bit to push your checkin-needed patches
• Level 3: full access, need 2 vouchers from module owners

mach

$python2.7 mach
usage: mach [global arguments] command [command arguments]

mach (German for "do") is the main interface to the Mozilla build system and common developer tasks.

Mozilla on OpenBSD

Not only firefox

• devel/xulrunner
• mail/mozilla-thunderbird{,-i18n}
• productivity/sunbird
• www/mozilla-firefox{,-i18n}
• www/seamonkey{,-i18n}
• www/firefox36
• www/fennec

History

• mozilla-19980728, not working
• 2002/2003: initial Mozilla 1.1 work, a.out only (jolan@)
• May 2003: Mozilla 1.4a/Firebird 0.6 runs (i386/sparc64/alpha) ! (wilfried@)
• Sept 2003: Flamewar on ports@ about which mozilla should be default
• January 2004: Thunderbird 0.4
• February 2004: Firefox 0.8
• End of 2005: Firefox 1.5 (bernd@)

History (cont'd)

• End of 2006: Seamonkey 1.0.7 (kurt@)
• March 2007: Firefox 2.0 (martynas@)
• May 2007: Mozilla 1.7 to the Attic/
• Mid-2008: Firefox 3.0
• June 2008: Firefox 3.5
• March 2010: Firefox 3.6 (landry@)
• March 2011: Firefox 4.0!

State in 2010

• TONS of local patches
• No comments in patches
• Maintainer MIA
• Duplicated stuff everywhere
• Diverging stuff everywhere
• I needed lightning in TB so...

Patchy patchy patch

Local customization vs upstreamable patch ?

mozilla.port.mk

• MODULES framework
• Abstracts common things between all 7 users
• ports Makefile greatly shrinked (- 90/100 lines)
• Sets common depends/configure options
• MOZILLA_PROJECT/MOZILLA_BRANCH
• Old stuff vs new stuff

ports in git

• http://rhaalovely.net/cgit/mozilla-firefox/
• My own workflow, ymmv
• Track CVS/ subdirs
• master = cvs
• release = last upstream stable/chemspill
• beta = last upstream beta
• do the cvs add/rm / branch merge dance

branches workflow in git

• Would have been better as a graph, but i suck at dia(1)
• Fx X.0bY released - work in beta
• merge beta to release when a final/candidate version is released
• merge release to master when commiting to cvs
• merge master to beta to stay in sync wrt cvs
• Fx X.0.Z chemspill - work in release, merge to master & to beta

updating a port

# Change MOZILLA_VERSION, adjust deps
make makesum
make patch
# fix/rm/add patches if needed
make update-patches
make update-plist
make port-lib-depends-check

Hardware

• amd64: sun v20z, opteron 252 2.6ghz / 4g (2h)
• i386: sun x2100, opteron 175 2.2ghz / 2g (2/3h)
• macppc: mac mini 1.5ghz / 1g (10 hours!)
• Xserve ppc g5 on the way (someday)
• sparc64 used to be a v240
• + desktops/laptops...
• Needed: beefier HW, moar RAM for mfs!

Buildbot

• http://buildbot.rhaalovely.net
• Dashboard keeping track of all builds w/ logs
• Pull hg tip, apply patches, configure, build, package, upload
• Nightly build of mozilla-aurora, mozilla-central & comm-central
• Produce a mozilla package directly usable for testing
• As close as possible from upstream builds
• Allows to find regression windows

Relationship with upstream

• They start knowing we're here
• Not priority platform, but try to not break it
• feedback? on important changes
• We need to do the work
• Upstream welcoming++ portability patches!!!

Relationship between *BSDs

• Know your alter ego
• CC people on patches sent upstream
• Cherrypick selected patches in other ports
• Fight for a common cause
• Could be improved...

Main problems

• Use of posix linux apis
• Header order inclusion
• Big code drops
• Need newer gcc or clang since Fx 17
• Memory hog when linking libxul.so
• Dependency on very latest sqlite/nss/nspr
• --with-system-XXX usually not-so-supported

State in 2013

• Up to date, closely following upstream
• ESR not considered
• Way less patches, much more documented
• Ports Makefiles sanitized
• Lack of feedback....
• What to do with 'old' things ?

Future work

• WebRTC (Fx 21 for tier1 plats)
• Improve HTML5 A/V via GStreamer
• Improve JS perfs (asm.js?)
• WebGL (works on intel!)
• Telemetry/FHR
• Breakpad/Profiler
• Resurrect sparc64 w/ martin@NetBSD awesome work
• Push back results from my buildbot to tbplv2

Questions ?

Slides on http://rhaalovely.net/~landry/bsdcan2013/

Thanks for listening