There is one issue that bazaars, library operating systems like MirageOS or random left-padists face: how to scale extreme software modularization. Scaling not from a software composition perspective; this could be readily solved by using the type system and the functional programming techniques everyone’s heard about. But scaling from a purely bureaucratic point of view.
The bureaucracy of making sure main issues are being taken care of. The bureaucracy of making sure changes to the software are being clearly communicated to dependents. The bureaucracy of making sure the software is correctly versioned and is able to precisely self-describe as such. The bureaucracy of making sure the software is a good citizen of the eco-system and that it follows its evolution. The bureaucracy of being able to quickly push fixes and let dependents test them as if they were released. The bureaucracy of making flawless releases: publishing correct and reproducible tarballs, publishing documentation and submitting the software package to the OCaml opam repository.
All this has clearly nothing to do with the fun of programming and calls for efficient, standardized, “don’t make me think” procedures and conventions. To give some perspective I myself formally publish 28 software packages, the MirageOS project directly manages 93 packages, any two minutes administrative task on each of these respectively represent one and three hours gone – dreadful hours. Any non-project specific bit that has to be committed to these repositories is maintenance burden and a curse.
This is where topkg fits in.
If you want to learn more about it, make sure to read the documentation
the tool manual (
opam install topkg-care && topkg help). To sum up,
topkg enough information so that it is able to determine what
needs to be installed in the current build environment. From these bits
topkg will infer targets to invoke in your build system – rather than
try to generate one – and produce a manifest file to let the package
manager install your package.
A few additional conventions and metadata bits you specify allow to
lint the package for good practices and release your software with the
topkg tool according to a well defined script. Read all the details
topkg help release but the gist of it is:
topkg browse issues # Review remaining outstanding issues topkg status # Review the changes since last version topkg log edit # Write the release notes topkg log commit # Commit the release notes topkg tag # Tag the distribution with a version topkg distrib # Create the distribution archive topkg publish # Publish it on the WWW with its documentation topkg opam pkg # Create an opam package topkg opam submit # Submit it to OCaml's opam repository The last four steps can be performed via a single invocation to topkg bistro.
topkg is far from being a perfect system, it tries to compose with
the existing build infrastructure. It is a step towards what is needed
from a packaging workflow perspective to scale modularity: the act of
releasing should be an inane, swift, robust and painless process.
If you are interested in programming and sharing your results, give topkg a try. No left-pad function left behind.
- Composable Builds for OCaml with Jbuilder
- MirageOS Hack Updates
- Algebraic Subtyping: PhD thesis
- Databox Launch Event: 24th March, Cambridge
- Formalizing the OCaml multicore memory model
- MirageOS Hack Event 2017 in Marrakech
- Irmin 1.0 Released!
- MirageOS 3.0.0 Released!
- Making PPXs portable with ocaml-migrate-parsetree
- Feedback on cmdliner doc language changes