mirage

Major Changes

• The solo5 targets ukvm and virtio (see https://mirage.io/blog/introducing-solo5), which allow unikernels to run on KVM hypervisors and on FreeBSD via bhyve, are now available in the mainline mirage package. These targets use a newly refactored freestanding OCaml runtime (see https://github.com/mirage/ocaml-freestanding).
• The V1 and V1_LWT modules, which contained module type definitions agreed upon by implementations used in mirage, have been renamed to Mirage_types and Mirage_types_lwt. The module type definitions themselves are no longer contained in this repository, but have been moved to their own independently-versioned repositories and are merely referenced from Mirage_types and Mirage_types_lwt. Users are encouraged to refer to the module types directly (e.g., Mirage_block_lwt.S instead of Mirage_types_lwt.BLOCK), and depend on the corresponding packages.
• Across the universe of module type definitions, errors in computation are now signalled by a result type and a set of polymorphic variants representing common errors that might occur. Individual implementations may extend this set, and users can always count on the provided module to provide a pretty-printer for these errors.
• The CLOCK module type has been replaced with PCLOCK and MCLOCK, which represent a POSIX-style wall clock and a monotonically-increasing clock respectively. The Mirage module provides default_posix_clock and default_monotonic_clock for getting an impl of each clock. Existing functions which took a clock impl argument now take an mclock impl wherever this is reasonable (in practice, everywhere except for logs).
• mirage-types-lwt is now its own opam package, rather than an optionally installed sub-library bundled with mirage-types.
• mirage configure no longer automatically attempts to install packages. Rather, it generates a .opam file representing the dependencies that have been discovered based on the config.ml and the arguments passed to mirage configure. Running make depend against the Makefile generated by mirage configure will attempt to install these dependencies via opam.
• Documentation for the project is now automatically built and provided via odig.
• The build system for mirage and many other libraries has been changed from OASIS to topkg.

Less Major API changes to Mirage module

• Mirage.register and Mirage.foreign no longer take a libraries argument. Their packages argument is now of type Functoria.package list, and the library information is included in package.
• connect functions provided by individual implementations are no longer expected to return a t result or Ok t by the mirage front-end tool's code generation. Rather, they are expected to raise an exception on failure and return the value directly on success.
• Two random impls are now available - stdlib_random, which wraps the OCaml Random module, and nocrypto_random, which is a passthrough to the Fortuna PRNG implemented in the nocrypto library. default_random is available for accessing stdlib_random by default, but can be told to use nocrypto_random at compile or runtime via the prng key.
• A syslog implementation is usable from mirage. Construct a record of type Mirage.syslog_config and call Mirage.syslog_udp, Mirage.syslog_tcp, or Mirage.syslog_tls to obtain a syslog impl.
• Functions for interfacing with the mirage-qubes library are now included: Mirage.qubes_ipv4_stack and Mirage.ipv4_qubes, as well as Mirage.default_qubesdb. Unikernels which are built with these functions will fail unless mirage configure is invoked with -t qubes.
• Mirage.tap0 has been renamed to Mirage.default_network and now works as expected on Xen targets.
• Mirage.farp is available as an alternative to Mirage.arp, and will use https://github.com/hannesm/arp instead of the ARP implementation in mirage-tcpip.
• The Mirage.ipv4_config and Mirage.ipv6_config types have been changed to reflect reasonable configuration assumptions rather than attempt to mirror one another's structure.
• Mirage.create_ipv4 now takes ethernet impl and arpv4 impl arguments, rather than creating an ipv4 impl directly from a network impl and hiding the intermediate ethernet and arpv4 from the caller.
• Mirage.create_ipv6 now takes an ethernet impl argument rather than a network impl argument.
• Mirage.default_ipv4 has been removed.
• Mirage.direct_udp now takes an optional random impl argument.
• Functions which yield a stackv4 impl now take an ipv4 impl argument instead of a network impl.
• Mirage.direct_stackv4_with_default_ipv4 and direct_stackv4_with_static_ipv4 no longer exist. Instead, consider using direct_stackv4, static_ipv4_stack, or dhcp_ipv4_stack.
• Network-related functions which took a console impl argument no longer do so.
• Mirage.get_mode, Mirage.add_to_opam_packages, and Mirage.add_to_ocamlfind_libraries have been removed (as warned in the 2.9.1 release).

Less Major API changes to Mirage_key module

• Mirage_key.is_xen has been removed and Mirage_key.is_unix added, as that is the more exceptional case.
• Mirage_key.ipv4 is now an (Ipaddr.V4.Prefix.t * Ipaddr.V4.t) converter. For an Ipaddr.V4.t converter, use the new Mirage_key.ipv4_address.
• A type Mirage_key.mode is available for referring to the set of polymorphic variants including Unix, Xen, MacOSX, Ukvm, Virtio, and Qubes.
• Mirage_key.no_ocaml_check has been removed.
• Mirage_key.kv_ro may now produce a direct key in addition to Archive, Crunch, and Fat keys.
• Mirage_key.prng is available for choosing between the standard library and nocrypto random implementations.
• Mirage_key.network has been renamed to Mirage_key.interface, as network has been repurposed in the Mirage_key.V4 and Mirage_key.V6 modules.
• Mirage_key.V4.ip and Mirage_key.V4.netmask have been replaced with Mirage_key.V4.network, which treats the IP and netmask as a single item.
• Mirage_key.V4.gateways has been replaced with Mirage_key.V4.gateway, which is a t option instead of the previous t list.
• Mirage_key.V6.ip and Mirage_key.V6.netmask have been replaced with Mirage_key.V6.ips and Mirage_key.V6.netmasks respectively, which accept lists instead of single items.

Less Major API changes outside of mirage

• format function and Format_unknown error have been removed from the FS module type in mirage-fs.
• read_stream, io_stream, and read_until have been removed from the CHANNEL module type in mirage-channel.
• log has been removed from the CONSOLE module type in mirage-console. log_s has been renamed to log to replace it.
• id has been removed from the DEVICE module type in mirage-device.
• The RANDOM module type provided by mirage-random now includes a function generate that provides random bytes as a buffer (currently a Cstruct.t), replacing the previous int-based interface.
• A mem function is now required by the KV_RO module type in mirage-kv.
• The read and size functions required by the KV_RO module type in mirage-kv now take int64s rather than ints.
• sleep (which took a float argument representing the number of seconds) has been replaced by sleep_ns (which takes an int64 representing the numer of nanoseconds) in the module type TIME provided by mirage-time.
• IP source and destination argument names have been standardized to src and dst, and source and destination ports to src_port and dst_port, in the module types provided by mirage-protocols.
• The pseudoheader function is now required for IP module types in mirage-protocols.

Less Major Usage Changes

• An additional target qubes exists for automatically packaging the additional libraries and initialization logic necessary for running MirageOS on QubesOS. The qubes target is at its heart xen plus some other stuff.
• mirage configure -f will no longer traverse into subdirectories. mirage configure needs to be called with a working directory the same as the configuration file.
• mirage configure --unix and mirage configure --xen are no longer understood, as warned in the 2.9.1 release. Instead, use mirage configure -t unix and mirage configure -t xen respectively.
• The artifact generated by make is no longer prefixed with mir-, and the name given to register will be stripped of OCaml-unfriendly characters before files are written.

Bugfixes and Other Improvements

• Functions used by mirage-generated programs at runtime have been split off into the mirage-runtime package, which will be linked instead of the full mirage package. (This is largely the collection of parsers for runtime keys.)
• The mirage command-line utility is now at mirage_cli.ml for easier discovery.
• It's now possible to get an ipv6 impl.
• Tar file generators for use with tar-format implementations are less broken.
• The Str module is no longer included in the OCaml runtime.
• Emit an ocamlfind predicate that matches the target.
• Improvements to nocrypto handling.
• Using crunch always requires io-page.
• Xen unikernels will no longer refuse to start when they receive unexpected boot parameters.
• Disable warning #42 in generated code.
• mirage-fs provides transforming FS into KV_RO, rather than requiring fat-filesystem.
• UDP now provides source port randomization even with the direct stack.
• DHCP is now provided by charrua-client and charrua-core rather than tcpip.
• Link libgcc.a only when building on ARM.
• Stop generating so many unnecessary artifacts.
• clean target removes more artifacts.
• Comply with mirage-clock-xen's rename to mirage-clock-freestanding.

• Warn users of command-line arguments --unix and --xen that support for
  these will soon be dropped. Instead, use -t unix and -t xen respectively.
  (see https://github.com/mirage/mirage-www/pull/475#issuecomment-233802501)
  (#561, by @yomimono)
• Warn users of functions add_to_opam_packages p and
  add_to_ocamlfind_libraries l that support for these will soon be dropped.
  Instead, use register ~libraries:l and register:~packages:p
  respectively. (#561, by @yomimono).

• Add logging support. A new reporter parameter to register is now
  available. This parameter defines how to configure the log reporter,
  using Logs and Mirage_logs. Log reporters can also be configured
  at configuration AND runtime using on the new -l or --logs
  command-line argument. (#534, by @samoht, @talex5 and @Drup)
• Allow to disable command-line parsing at runtime. There is a new
  argv parameter to the register function to allow to pass custom
  command-line argument parsing devices. Use register ~argv:no_argv
  to disable command-line argument parsing. (#493, by @samoht and @Drup)

• Fix another regression introduced in 2.7.1 which enable
  -warn-error by default. This is now controlled by a
  --warn-error flag on mirage configure. Currently it's
  default value is [false] but this might change in future
  versions (#520)

• Fix regression introduced in 2.7.1 which truncates the ouput of
  opam install and breaks opam depext (#519, by @samoht)

• Define an ICMP and ICMPV4 module type. ICMPV4 is included in, and surfaced by, the STACKV4 module type. The previous default behavior of the IPv4 module with respect to ICMP is preserved by STACKV4 and the tcpip_stack_direct function provided by mirage. (#523, by @yomimono)
• Explicitly require OCaml compiler version 4.02.3 in opam files for mirage-types and mirage.

• Improve the Dockerfile (#507, by @avsm)
• Use Astring (by @samoht)
• Clean-up dependencies automatically added by the tool
  • do not require lwt.syntax, cstruct.syntax and sexplib, which
    should make the default unikernels camlp4-free (#510, #515 by @samoht)
  • always require mirage-platform (#512, by @talex5)
  • ensure that mirage-types >= 2.6.0 and mirage-types-lwt are
    installed
• Check that the OCaml compiler is at least 4.02.3 (by @samoht)

The mirage tool is now based on functoria.
See https://mirage.io/blog/introducing-functoria for full details.

• Command line interface: The config file must be passed with the -f option
  (instead of being just an argument).

• Two new generic combinators are available, generic_stack and generic_kv_ro.

• get_mode is deprecated. You should use keys instead. And in particular
  Key.target and Key.is_xen.

• add_to_ocamlfind_libraries and add_to_opam_packages are deprecated. Both
  the foreign and the register functions now accept the ~libraries and
  ~packages arguments to specify library dependencies.

• If you were using tls without the conduit combinator, you will be
  greeted during configuration by a message like this:

  The "nocrypto" library is loaded but entropy is not enabled!
  Please enable the entropy by adding a dependency to the nocrypto device.
  You can do so by adding ~deps:[abstract nocrypto] to the arguments of Mirage.foreign.
  

  Data dependencies (such as entropy initialization) are now explicit.
  In order to fix this, you need to declare the dependency like so:

  open Mirage
  
  let my_functor =
  let deps = [abstract nocrypto] in
  foreign ~deps "My_Functor" (foo @-> bar)
  

  My_functor.start will now take an extra argument for each
  dependencies. In the case of nocrypto, this is ().

• Remove nat-script.sh from the scripts directory, to be available
  as an external script.

• Xen: improve the .xl file generation. We now have
  • name.xl: this has sensible defaults for everything including the
    network bridges and should "just work" if used on the build box
  • name.xl.in: this has all the settings needed to boot (e.g. presence of
    block and network devices) but all the environmental dependencies are
    represented by easily-substitutable variables. This file is intended for
    production use: simply replace the variables for the paths, bridges, memory
    sizes etc. and run xl create as before.

• Better ARP support. This needs mirage-tcpip.2.6.0 (#419, by @yomimono)
  • [mirage-types] Remove V1.IPV4.input_arp
  • [mirage-types] Expose V1.ARP and V1_LWT.ARP
  • Expose a Mirage.arp combinator
• Provide noop configuration for default_time (#435, by @yomimono)
• Add Mirage.archive and Mirage.archive_of_files to support attaching files
  via a read-only tar-formatted BLOCK (#432, by @djs55)
• Add a .merlin file (#428, by @Drup)

• [mirage-types] Expose V1_LWT.FS.page_aligned_buffer = Cstruct.t

• Change the type of the Mirage.http_server combinator. The first argument
  (the conduit server configuration) is removed and should now be provided
  at compile-time in unikernel.ml instead of configuration-time in
  config.ml:

  (* [config.ml] *)
  (* in 2.4 *) let http = http_server (`TCP (`Port 80)) conduit
  (* in 2.5 *) let http = http_server conduit
  
  (* [unikernel.ml] *)
  let start http =
  (* in 2.4 *) http (S.make ~conn_closed ~callback ())
  (* in 2.5 *) http (`TCP 80) (S.make ~conn_closed ~callback ())
  

• Change the type of the Mirage.conduit_direct combinator.
  Previously, it took an optional vchan implementation, an optional
  tls immplementation and an optional stackv4 implemenation. Now,
  it simply takes a stackv4 implementation and a boolean to enable
  or disable the tls stack. Users who want to continue to use
  vchan with conduit should now use the Vchan functors inside
  unikernel.ml instead of the combinators in config.ml. To
  enable the TLS stack:

  (* [config.ml] *)
  let conduit = conduit_direct ~tls:true (stack default_console)
  
  (* [unikernel.ml] *)
  module Main (C: Conduit_mirage.S): struct
  let start conduit =
    C.listen conduit (`TLS (tls_config, `TCP 443)) callback
  end
  

• [types] Remove V1.ENTROPY and V1_LWT.ENTROPY. The entropy is now
  handled directly by nocrypto.0.4.0 and the mirage-tool is only responsible to
  call the Nocrypto_entropy_{mode}.initialize function.

• Remove Mirage.vchan, Mirage.vchan_localhost, Mirage.vchan_xen and
  Mirage.vchan_default. Vchan users need to adapt their code to directly
  use the Vchan functors instead of relying on the combinators.

• Remove Mirage.conduit_client and Mirage.conduit_server types.

• Fix misleading "Compiling for target" messages in mirage build
  (#408 by @lnmx)

• Add --no-depext to disable the automatic installation of opam depexts (#402)

• Support @name/file findlib's extended name syntax in xen_linkopts fields.
  @name is expanded to %{lib}%/name

• Modernize the Travis CI scripts

• Support mirage-http.2.2.0
• Support conduit.0.8.0
• Support tcpip.2.4.0
• Add time and clock parameters to IPv4 (#362, patch from @yomimono)
• Support for ocaml-tls 0.4.0.
• Conduit now takes an optional TLS argument, allowing servers to support
  encryption. (#347)
• Add the ability to specify Makefile.user to extend the generated
  Makefile. Also all, build and clean are now extensible make
  targets.
• Remove the mirage run command (#379)
• Call opam depext when configuring (#373)
• Add opam files for mirage and mirage-types packages
• Fix mirage --version (#374)
• Add a update-doc target to the Makefile to easily update the online
  documentation at http://mirage.github.io/mirage/

• Remove the IO_PAGE module type from V1. This has now moved into the
  io-page pacakge (#356)
• Remove DEVICE.connect from the V1 module types. When a module is
  functorised over a DEVICE it should only have the ability to
  use devices it is given, not to connect to new ones. (#150)
• Add FLOW.error_message to the V1 module types to allow for
  generic handling of errors. (#346)
• Add IP.uipaddr as a universal IP address type. (#361)
• Support the entropy version 0.2+ interfaces. (#359)
• Check that the opam command is at least version 1.2.0 (#355)
• Don't put '-classic-display' in the generated Makefiles. (#364)

• Fix logging errors when mirage output is not redirected. (#355)
• Do not reverse the order of C libraries when linking. This fixes Zarith
  linking in Xen mode. (#341).
• Fix typos in command line help. (#352).

Add IPv6 support, from Nicolas Ojeda Bar. This alters some of the interfaces that were previously hardcoded to IPv4 by generalising them. For example:

type v4
type v6

type 'a ip
type ipv4 = v4 ip
type ipv6 = v6 ip

Full support for configuring IPv6 does not exist yet, as this release is
intended for getting the type definitions in place before adding configuration
support.

• Do not reuse the Unix linker options when building Xen unikernels. Instead,
  get the linker options from the ocamlfind xen_linkopts variables (#332).
  See tcpip.2.1.0 for a library that does this for a C binding.
• Only activate MacOS X compilation by default on 10.10 (Yosemite) or higher.
  Older revisions of MacOS X will use the generic Unix mode by default, since
  the vmnet framework requires Yosemite or higher.
• Do not run crunched filesystem modules through camlp4, which significantly
  speeds up compilation on ARM platforms (from minutes to seconds!) (#299).

• Add specific support for MacOSX as a platform, which enables network bridging
  on Yosemite (#329). The --unix flag will automatically activate the new target
  if run on a MacOS X host. If this breaks for you due to being on an older version of
  MacOS X, then use the new --target flag to set either Unix, MacOSX or Xen to the
  mirage configure command.
• Add mirage.runtime findlib library and corresponding Mirage_runtime module (#327).
• If net driver in STACKV4_direct can't initialize, print a helpful error (#164).
• [xen]: fixed link order in generated Makefile (#322).
• Make Lwt.tracing instructions work for Fish shell too by improving quoting (#328).

• Add register ~tracing to enable tracing with mirage-profile at start-up (#321).
• Update Dockerfile for latest libraries (#320).
• Only build mirage-types if Io_page is also installed (#324).

Backwards incompatible changes to V1 types:

• CONSOLE is now a FLOW, so write has a different signature and 'write_all' has been removed.

New features in the CLI and config parser:

• Set on_crash = 'preserve' in default Xen config.
• Automatically install dependencies again, but display the live output to the user.
• Include C stub libraries in linker command when generating Makefiles for Xen.
• Add Vchan, Conduit and Resolver code generators.
• Generate a *.xe script which can upload a kernel to a XenServer.
• Generate a libvirt *.xml configuration file (#292).
• Fix determination of mirage-xen location for paths with spaces (#279).
• Correctly show config file locations when using a custom one.
• Fix generation of foreign (non-functor) modules (#293)