Mark Bixby csy r&d lab august 22, 2001 open-source development on mpe August 22, 2001 HP World Page 1
•a software package for which the source code is freely available definition of open source August 22, 2001 •possible modification restrictions •possible redistribution restrictions •many differing license variations, i.e.
•one porter can leverage the efforts of hundreds of open source developers benefits of open source on the e3000 •easy way to add leading-edge functionality to the platform •popular open source apps are defacto standards •easier to hire people knowledgeable about open source apps than it is to hire people knowledgeable about MPE August 22, 2001 HP World Page 3
•1971 - what would become the Interex Contributed Software Library is born •1972 - the HP 3000 is born history - in the beginning •1977 - first conference software swap tape is produced •1982 - last cumulative CSL tape; switch to "best of" + incremental •1988 - CSL switches to incremental-only format •1990 - CSL split into separate MPE and MPE/XL distributions August 22, 2001 HP World Page 4
•1992 - MPE/iX 4.5 arrives with support for POSIX 1003.1 (API) and 1003.2 (shell & utilities). This voluntary release achieved limited exposure, primarily in the ISV community. history - posix arrives •1994 - MPE/iX 5.0 "Pull" arrives with more reliable POSIX support. Customers interested in this release had to explicitly request it. •1995 - MPE/iX 5.
•1994 - NCSA web server, Lynx web browser, various BSD functionality ported by Mike Belshe & Steve Elmer •1995 - GNU gcc compiler & related tools ported by Mark Klein history - proliferation •1996 - Samba ported by Lars Appel •1997 - Apache, Syslog, BIND, Perl, NTP ported by Mark Bixby •1998 - Sendmail ported by Mark Bixby •2001 - PostgreSQL RDBMS ported by Mark Bixby ...and many more ports and porters http://jazz.external.hp.
•1998 - Syslog, BIND, Samba •1999 - Apache history - productization •2000 - HP WebWise MPE/iX Secure Web Server •2001 - Sendmail (announced) ...
•cryptography the good - apps that port easily •compression •socket networking •graphic manipulation •many GNU utilities August 22, 2001 HP World Page 8
the bad - ports unlikely to succeed August 22, 2001 •select()-based terminal I/O •pseudo TTYs •multicasting sockets HP World Page 9
the ugly - use extra care porting these kinds of apps because "here be dragons" August 22, 2001 •threads (though threads are slowly improving on MPE!) •dynamic loading •heavy dependence on root uid 0 or setuid() •reliance on link() and hard links HP World Page 10
•maximum socket read/write size is 30000 bytes •some functions like link() exist but return errno == EIMPL when called •use telnet instead of VT in the shell to avoid heavy CTRL-Y overhead; set +o vi for more improvement hidden pitfalls •ld linker doesn't flag unresolved externals by default; to enable this, set the AUTOCONF environment variable to 1 •built-in cat command can lose bytes on "here docs" -- use /bin/cat instead, i.e.
•you can call intrinsics instead •gcc doesn't understand "#pragma intrinsic", so you will need to declare all parameters including any hidden ones when posix is not enough •CSEQ can generate intrinsic declarations for gcc http://www.lund.com/Products/De vTool/index.html •#include for ccode() and _mpe_fileno() •#include for _MPE_FILENO() and _POSIX_FILENO() •gcc needs help for long pointers http://raven.utc.edu/cgibin/WA.
•POSIX functions requiring root uid 0 on Unix require priv-mode on MPE priv-mode – bind() to ports < 1024 – setuid() •extern GETPRIVMODE(); •extern GETUSERMODE(); •PM NMPRGs and NMXLs *must* reside in MPE namespace August 22, 2001 HP World Page 13
•Choose an application you're already familiar with •Have the following manuals ready: • MPE/iX Developer's Kit Reference Manual Vol I&II • HP C/iX Library Reference Man.
•No! Most open source apps have already been designed to make porting easy do I need to be a C or unix expert? •95-100% of most apps will work as-is on MPE •Almost all of the problems you might encounter have already been solved by other MPE porters; if you get stuck, just ask for help on HP3000-L •Open source is not just C; it could be Java or Perl or Python or ...
•Most apps use a GNU autoconf script to evaluate the build OS to determine which include files and functions exist configure •GNU autoconf assumes ld will report unresolved externals which does not happen by default on MPE; to enable this: – AUTOCONF=1 configure parms... – or regenerate configure from configure.
•Almost every open source app uses the make command and Makefiles for compiling compile •Makefiles describe dependency relationships and the rules for compiling •Just type "make" and everything with missing or out-of-date object code will be compiled August 22, 2001 HP World Page 17
•The most common compile error is for missing #include files, of which several really aren't needed on MPE •If in doubt, try commenting it out! compile (cont.) •#ifndef MPE /* generic */ #include #endif •#ifdef HAVE_SYS_PARAM_H #include
•Compile-time options determine how #include files are parsed •You will always need -D_POSIX_SOURCE compile (cont.) •Socket programs will always need -D_SOCKET_SOURCE •You may very infrequently also need -D_MPEXL_SOURCE •You can usually specify these at configure time, i.e.: CPPFLAGS="-D_POSIX_SOURCE" \ configure parms...
•The moment of truth for unresolved externals! Try running every program with bogus parameters to check for unresolved externals.
•If the app has a test suite, run it and investigate all failures test •Perform your own MPE-specific testing on any logic that you had to alter during the porting process •Test the key features that you think MPE users are most likely to need •Consider releasing an alpha test distribution, but be prepared for low participation August 22, 2001 HP World Page 21
•For simple apps, a tar or :STOREto-disk archive may suffice •Complex apps may require a custom INSTALL script to create the proper MPE accounting structure package •Think about future updates -- do you want to allow multiple versions on the same machine at the same time? •If you're thinking of doing multiple ports, spend the time on your first port designing a common installation architecture template •Test your package by installing and/or building it on a machine different from your own August 22, 200
•You are NOT finished until you have submitted your MPE changes back to the original developers for inclusion into future releases!!! submit •When making porting changes, copy the same look & feel from changes supporting other OSes •Submit your changes in GNU diff format, i.e.
software tools • GNU gcc bundle - a C/C++ compiler and a whole lot more • http://jazz.external.hp.com/src/gnu/gnuframe.html • commercial support available from http://www.gccsupport.com/ • Porting Wrappers - implementations of various missing functions • http://jazz.external.hp.com/src/px_wrappers/index.html • unsupported freeware • Porting Scanner - analyzes source and binaries to determine effort • http://jazz.external.hp.com/src/ps/index.
knowledge resources • Various MPE manuals from http://docs.hp.com/mpeix/all/ • MPE/iX Developer's Kit Reference Manual Vol 1&2 • MPE/iX Shell and Utilities Reference Manual, Vol 1&2 • HP C/iX Library Reference Manual • HP C/iX Reference Manual • Berkeley Sockets/iX Reference Guide • Mark Bixby's MPE/iX Porting Guide • http://www.bixby.org/mark/porting.html • Lars Appel's HPe3000 Page • http://www.editcorp.com/Personal/Lars_Appel/index.
•Register for Internet access to a fully-loaded HP e3000 989-400 system •Port or develop new open source software csy public access development system •Develop new closed source software •Test software on the newer releases of MPE •Support not included •For full details, please see http://jazz.external.hp.
•Open source works best as a community effort •The original porters will not feel offended if somebody else wants to build on their MPE ports partnership •CSY welcomes community involvement in enhancing HPsupported open source products •Your porting efforts allow CSY to adopt key infrastructure technologies as supported products August 22, 2001 HP World Page 27
•an updated and supported version of Sendmail the future •periodic updates as needed of Apache, Samba, WebWise and other key open-source based products •investigate fixing certain highvisibility port-enabling issues August 22, 2001 HP World Page 28
join the hp3000-L community! • Available as a mailing list and as the Usenet newsgroup comp.sys.hp.mpe • In-depth discussions of all things HP e3000 • Talk with other people involved with open source and porting on MPE • seek advice, exchange tips & techniques • Keep up with the latest HP e3000 news • Interact with CSY • http://jazz.external.hp.com/papers/hp3000-info.
