Membrane Core v0.11.0 is now available!

This release includes mostly API improvements. For some of them, breaking changes were necessary. To facilitate migration, we prepared the updating guide that should help adjust your code to the changes.

Breaking changes

Actions

The following actions have changed:

• Membrane.Element.Action.caps_t -> Membrane.Element.Action.stream_format_t (#468)
• Membrane.Bin.Action.notify_t -> Membrane.Bin.Action.notify_parent_t (#415)
• Membrane.Element.Action.notify_t -> Membrane.Element.Action.notify_parent_t (#415)
• Membrane.Pipeline.Action.forward_t -> Membrane.Pipeline.Action.notify_child_t (#415)
• Membrane.Bin.Action.forward_t -> Membrane.Bin.Action.notify_child_t (#415)

Callbacks

From now on, the callbacks should return an {actions, state} tuple. Returning errors is no longer supported - if a component can't recover from an error, it should raise (#471)

Furthermore, the following callbacks have changed:

• handle_stopped_to_prepared/2 -> handle_setup/2 in all components (#432)
• Membrane.Element.Base.handle_caps/4 -> Membrane.Element.Base.handle_stream_format/4 (#432)
• handle_element_start_of_stream/3 -> handle_element_start_of_stream/4 and handle_element_end_of_stream/3 -> handle_element_end_of_stream/4 in pipelines and bins (#417)
• handle_init/1 -> handle_init/2 in all components (#432)
• handle_other/3 -> handle_info/3 in all components (#415)
• handle_notification/4 -> handle_child_notification/4 in pipelines and bins (#415)

The following callbacks have been removed:

• handle_playing_to_prepared/2 (#432)
• handle_prepared_to_stopped/2 (#432)
• handle_shutdown/2(#432)

Instead, Membrane.ResourceGuard, Membrane.UtilitySupervisor or handle_terminate_request/2 should be used.

What is more, messages sent with the :notify_parent action (former :notify action), are handled in a separate handle_parent_notification/3 callback instead of being handled in handle_other/3, along with messages sent from any other processes. (#415)

Pads definitions

In v0.11 :caps option passed to def_input_pad/2 or def_output_pad/2 has been deleted. Instead of it, we have added a new option - :accepted_format. The main principle of these two options is the same - validation of value passed to :caps or :stream_format action. While :caps required a module name or specific tuple format as argument, :accepted_format requires following types of terms:

• Elixir pattern - eg. accepted_format: %My.Custom.Format{field: value} when value in [:some, :enumeration]

  The value passed to :stream_format action has to match the provided pattern. In this case, the requirement above would
  be satisfied by eg. stream_format: %My.Custom.Format{field: :some}

• Module name - eg. accepted_format: My.Custom.Format
  This would be equal to the match on the struct of the passed module, in this case accepted_format: %My.Custom.Format{}

• Call to any_of function - you can pass as many arguments to it, as you want. Each argument should be an Elixir pattern
  or a module name, eg. stream_format: any_of(My.Custom.Format, %My.Another.Custom.Format{field: :value})

  If you use any_of, the value passed to :stream_format will have to match at least one of the passed
  arguments. In this case, stream_format: %My.Custom.Format{frequency: 1} would be ok, but
  stream_format: %My.Another.Custom.Format{field: :not_allowed_value} would fail

Option :accepted_format is required. If you don't want to perform any check on stream_format, you can always write accepted_format: _any, but it is not suggested.

Checks on stream_format will be performed on both, intput and output pads, just as caps were checked in those places.

New way to define children

• Membrane.Bin.spec_t and Membrane.Pipeline.spec_t actions no more accept Membrane.ParentSpec structure.
  Instead, a tuple with the children structure and options needs to be passed. (#458)
• Functions used to spawn children and create links between them, available in the former Membrane.ParentSpec module, have been changed - till now on, the children structure needs to be defined with the use of child/3, child/4, get_child/2 and get_child/3 functions from the Membrane.ChildrenSpec. (#458)
• Membrane.ParentSpec.link_bin_input/1 has been renamed to Membrane.ChildrenSpec.bin_input/1 (#458)
• Membrane.ParentSpec.link_bin_output/2 has been renamed to Membrane.ChildrenSpec.bin_output/2 (#458)

Changes in Membrane.Time module

• Membrane.Time.to_<unit name>/1 has been renamed to Membrane.Time.round_to_<unit name>/1 in order to indicate that the result will be rounded. (#435)

Changes with def_options macro

• A :type field from the def_options keyword list has been removed, and the type specification of the option defined within the macro is determined by the :spec field (#466)

New way to spawn pipeline

• Membrane.Pipeline.start/3 and Membrane.Pipeline.start_link/3 now return {:ok, pipeline_supervisor, pipeline}.(#432)

Changes in Membrane.Testing

• Membrane.Testing.Pipeline.start_link/1 has been changed to Membrane.Testing.Pipeline.start_link_supervised!/1 (#432)
• The Membrane.Testing.Pipeline.options() now have a single :structure field allowing to specify the test pipeline's children structure, in place of the former :links and :children fields.

Other changes:

• handle_call/3 callback in the pipeline and areply and :reply_to actions. (#334)
• Improvements in documentation - till now on, in the documentation page for each of the following modules: Membrane.Source, Membrane.Filter, Membrane.Endpoint and Membrane.Sink there is a list of all the callbacks available to be implemented for these modules.
• Membrane.Time.<plural unit name>/1 now accepts Ratio structure as an argument. (#435)
• Membrane.Time.round_to_timebase/2 function has been added. (#435)
• Membrane.FilterAggregator that allows running multiple filters sequentially within one process has been added (still experimental) (#355)
• Information about the element's playback change is logged as debug, not as debug_verbose. (#430)
• Membrane.Testing.MockResourceGuard has been added, to help write tests with the new way of handling resources. (#478)

What's changed?

Changes that do not affect API

• A bug with the distributed pipeline crashing has been fixed. The pipeline which elements were put on different nodes used to crash due to the fact, that the toilet between the elements was designed to be used by elements running on the same node.

What's changed?

The changes that do not affect API:

• improvements in documentation and exemplary code snippets, concerning the latest changes in API
• the errors in Membrane's core are now raised as exceptions as low as they occur, instead of being propagated up with {:error, ...} monads
• mechanism for checking if the static pads are unlinked only when the element dies, has been turned off for the further investigation since it's not working properly
• bug fix: Membrane.Testing.Pipeline with a custom module injected behaves in a desired way once Membrane.Testing.Pipeline.execute_actions/2 function is called (the custom module does not need to implement handle_other({:execute_actions, actions}, ...) anymore)

Full changelog: v0.10.0 ... v0.10.1

Release includes:

• removing deprecated stuff #399:
  • Support for returning bare Membrane.ParentSpec{} from handle_init/1
  • Support for callbacks without context argument in Pipeline and Bin
  • Membrane.Element.WithInputPads.def_input_pads/1
  • Membrane.Element.WithOutputPads.def_output_pads/1
  • Membrane.Log.*
  • Membrane.ParentSpec.link_bin_input/2
  • Membrane.ParentSpec.to_bin_output/2
  • Following functions inMembrane.Time: is_t/1, system_time/0, native_unit/1, day/1, hour/1, minute/1, second/1, millisecond/1, microsecond/1, nanosecond/1
• making Membrane.Pipeline.{prepare, play, stop} deprecated and adding :playback action instead
• making Membrane.Pipeline.stop_and_terminate/2 deprecated and adding Membrane.Pipeline.terminate/2 instead
• adding the Membrane.RemoteControlled.Pipeline - a basic implementation of a Membrane.Pipeline that
  can be spawned and controlled by an external process #366
• disallowing sending buffers through pads without sending the caps first, by raising an Membrane.ElementError exception if such a situation occurs #341
• refining the Membrane.Testing.Pipeline API - deprecating the Membrane.Testing.Pipeline.Options usage, allowing to use pipeline_keyword_list_t as options in Membrane.Testing.Pipeline.start/1 and Membrane.Testing.Pipeline.start_link/1

Full changelog: v0.9.0 ... v0.10.0

Automatic demands

The Membrane Core can now handle the demands in filters for you! 🎉
To enable this, use demand_mode: :auto in your pad's definition:

def_input_pad :input,
    demand_mode: :auto,
    caps: :any

This will make the Core send a demand automatically, no need to use :demand action anymore.
By adding the same option to the output pad, the Core will also handle any incoming demands and demands on all input pads, so it is suitable for cases when a filter consumes data from all inputs uniformly.

See how automatic demands are used in funnel

What's Changed

• Automatic demands #313
• Stop forwarding notifications by default in bins #358
• More fine-grained control over emitted metrics #365
• Added log metadata when reporting init in telemetry #376
• Fix generation of pad documentation inside an element #377
• Leaving static pads unlinked and transiting to a playback state other than :stopped will result
  in runtime error (previously only a warning was printed out). #389
• It is possible now to assert on crash group down when using Testing.Pipeline. #391

Full Changelog: https://github.com/membraneframework/membrane_core/compare/v0.8.2...v0.9.0

Release includes:

• Fixed PadAdded spec #359
• Prevent internal testing notifications from reaching pipeline module #350
• Fix unknown node error on distribution changes #352

Release includes:

• Adds the possibility to specify Membrane.Logger metadata in the Membrane.ParentSpec. All children spawned from that Membrane.ParentSpec will receive specified metadata.
• PTS and DTS timestamps were added to Membrane.Buffer structure explicitly. Timestamps should no longer live in Membrane.Buffer.metadata field #335.
• Less warnings on shutdown when parent or other element crashed #322
• Added more telemetry events: #317
• Description of Element's options can now use interpolation and function calls #328

Crash Groups

From v0.7.0 of membrane_core it's possible to group elements and bins into crash groups.
Crash group is a logical entity that prevents the whole pipeline from crashing when one of its children crashes.

Adding children to the Crash Group

children = %{
  {:endpoint_bin, endpoint_id} => %EndpointBin{
    # ... 
  }
}

spec = %ParentSpec{children: children, crash_group: {endpoint_id, :temporary}}

In this case we create a new children - EndpointBin and we add it to crash group with id endpoint_id. When EndpointBin crashes the whole pipeline will still be alive.

Handling crash of Crash Group

When some child in a crash group crashes the callback handle_crash_group_down/3 is called.

@impl true
def handle_crash_group_down(crash_group_id, ctx, state) do
  Membrane.Logger.info("Crash group: #{inspect(crash_group_id)} is down.")
  # do some stuff 
end

Limitations

At this moment Crash Groups are only useful for elements with dynamic pads.

Condidional Linking

In this version we also introduce an enhancement in linking elements.
Sometimes there is a situation you want to link or create an element only when some requirements are met.
To make it easier we introduce a new syntax that looks like below

encoding_specific_links =
  case encoding do
    # when encoding is :H264 we want to add one additional element -- H264 parser.
    :H264 -> &to(&1, {:h264_parser, ssrc}, %Membrane.H264.FFmpeg.Parser{alignment: :nal}) 
    # when encoding is :OPUS we just passes what we got 
    :OPUS -> & &1 
  end

links = [
  link_bin_input(pad)
  |> pipe_fun(encoding_specific_links)
  |> to({:track_filter, track_id}, %Membrane.WebRTC.TrackFilter{enabled: track_enabled})
  # ...
]

defp pipe_fun(term, fun), do: fun.(term)

Breaking changes

From now Testing.Source sends a new type of caps - RemoteStream. This can break some of your already existing tests.

This release includes several bug fixes:

• bin pad opts handling #282
• docs generation #283
• children playback state management #284

Breaking changes

• Callbacks in pipelines and bins now have the context argument #278
• [Fix] Remove stale Membrane.Event.{StartOfStream, EndOfStream} events. Since v0.5.0 they're replaced with dedicated action and callbacks #280

Other changes

• Bins and pipelines can now use timers #264
• forward action in pipelines and bins now accepts list #279
• Docs fixes #280
• Started integration with telemetry #269

• fix stopping timers bug #268
• update CI docker image

• Fix RC when terminating #252 (turned out not to be fully fixed by v0.5.1)
• Fix warnings on Elixir 1.10 #241
• Add NTP timestamp conversion to Membrane Time #248

• Improved log pruning, other performance improvements #246, #244
• Fix in timer time conversion algorithm #226
• Fixes related to the end of stream generation #216
• Lots of tests, docs and type specs improvements
• Synchronous pipeline termination #233
• timer_interval action #222
• New Membrane.Time as_* functions #223

• Fix missing key in InputBuffer warning
• Fix log pruning

Membrane Framework 0.5 release notes

Links

Although the main feature of this release is the introduction of bins, let's start with the refreshed syntax for links, as we'll need them for bins also. Until now, the way you declared links was the following:

links = %{
   {:file_src, :output} => {:decoder, :input},
   {:decoder, :output} => {:converter, :input},
   {:converter, :output} => {:player, :input}
 }

That was a bit too verbose and made trouble when linking a dynamic output pad multiple times, causing repeated keys in the map. The new way solves both problems:

links = [
  link(:file_src) |> to(:decoder) |> to(:converter) |> to(:player)
]

If a pipeline is not linear, more elements can be added to the list:

links = [
  link(:microphone) |> to(:tee) |> to(:player),
  link(:tee) |> to(:file)
]

As :output and :input are default pad names, there is no need to provide them explicitly. If a custom name or pad options should be passed, via_out/2 and via_in/2 functions come to help.

links = [
  link(:element1)
  |> via_out(:custom_output, options: [key: :value])
  |> via_in(:custom_input)
  |> to(:element2)
]

Also, the way dynamic pads are referenced has been unified. Now, instead of passing a pad name and id to a link, full reference should be passed. As the existing format of references {:dynamic, name, id} would be a bit cumbersome for that, it is now handled with Membrane.Pad.ref(name, id) macro.

links = [
  link(:file_src) |> via_in(Pad.ref(:input, 3)) |> to(:mixer)
]

See the docs for Membrane.ParentSpec for more details.

Bins

Bins - entities that enable creating reusable, dynamically customisable groups of elements, are now available in Membrane. Bins can spawn their children like pipelines and have pads like elements, so they can be embedded within a pipeline (or another bin). Bins' pads proxy the stream between their siblings and children.

To use a bin, add it to children and link the same way as elements:

children = [
  element1: Element1,
  my_bin: My.Bin,
  element2: Element2
]

links = [
  link(:element1) |> to(:my_bin) |> to(:element2)
]

When linking children within a bin, you can connect them to bin's pad with use of link_bin_input/2 and to_bin_output/3 from Membrane.ParentSpec.

links = [
  link_bin_input() |> to(:child1) |> to(:child2) |> to_bin_output(:custom_output)
]

Bins can also have dynamic pads, and the new way of referencing them is intended to make this powerful blend straightforward to reason about.

@impl true
def handle_pad_added(Pad.ref(:input, _) = pad, _ctx, state) do
  links = [link_bin_input(pad) |> to(:mixer)]
  {{:ok, spec: %ParentSpec{links: links}}, state}
end

For more about bins check out the guide chapter about bins, see how to create and deal with bins in the documentation for Membrane.Bin, or have a look at RTP bin or RTP demo.

Breaking changes

• Membrane.Pipeline.Spec is now Membrane.ParentSpec
• Dynamic pads are now referenced by Membrane.Pad.ref(name, id) macro instead of {:dynamic, name, id} tuple
• Links are specified in a new way, described in docs for Membrane.ParentSpec
• Field pipeline_clock became parent_clock in all the callback contexts

This release fixes warning printed when pipeline is stopped via stop_and_terminate function.

• A fix for #185 - bug that prevented using monitors from elements
• Fix a crash when using start_timer without explicitly passing a clock (#195)
• Fix synchronization of elements spawned dynamically (after init) (#198)
• Other minor improvements in docs, tests and typespecs

Membrane Framework 0.4 release notes

Synchronization

The main feature of this release is synchronization utilities: clocks, timers and stream synchronization.

Stream synchronization

We added a new field in Membrane.Pipeline.Spec - stream_sync that accepts either atom :sinks or a list of lists containing elements' names.

%Spec{stream_sync: [[:element1, :element2], [:element3, :element4]]}

%Spec{stream_sync: :sinks}

See this guide chapter for more info.

In addition, each element can declare its latency affecting how the synchronization works. Element declaring 100 ms of latency will be started (will receive start of stream event) 100 ms earlier than element without latency.
Latency can be declared by returning :latency action from handle_init callback.

  @impl true
  def handle_init(options) do
    state = options |> Map.from_struct()
    {{:ok, latency: 20 |> Membrane.Time.milliseconds()}, state}
  end

See this guide chapter

Clocks

Elements may provide a different source of time - like a hardware clock from sound card or elapsed time according to some library. This release introduces a def_clock macro that can be used in elements exposing a clock.
The clock process spawned by the framework for such element will expect :membrane_clock_update messages containing a number of milliseconds until the next update.

A detailed guide on how to expose custom clock from an element can be found here

Timers

Since this release you can use timers started and stopped by 2 new actions: :start_timer and :stop_timer.

@impl true
def handle_start_of_stream(:input, ctx, state) do
  {{:ok, start_timer: {:demand_timer, 10}}, state}
end

Timers send ticks in intervals configured when a timer is started. The ticks can be handled by a new callback -
handle_tick/3. The details can be found in guide

@impl true
def handle_tick(:demand_timer, _ctx, state) do
  {{:ok, demand: :input}, state}
end

Testing API

Testing Pipeline

This Pipeline was created to reduce testing boilerplate and ease communication with its elements. It also provides a utility for informing the testing process about playback state changes and received notifications.

When you want a build Pipeline to test your elements you need three things:

• Pipeline Module
• List of elements
• Links between those elements

When creating pipelines for tests the only essential part is the list of elements. In most cases during the tests, elements are linked in a way that the :output pad is linked to the :input pad of subsequent element. So we only need to pass a list of elements and links can be generated automatically

{:ok, pipeline} = Membrane.Testing.Pipeline.start_link(
     %Membrane.Testing.Pipeline.Options {
         elements: [
            source: %Membrane.Testing.Source{},
            tested_element: TestedElement,
            sink: %Membrane.Testing.Sink{}
          ]
    }
)

In addition to that this module contains utilities like message_child/3 and populate_links/1

You can also pass a custom pipeline module, by using :module field of Membrane.Testing.Pipeline.Options struct. Every callback of the module will be executed before the callbacks implemented by Membrane.Testing.Pipeline. Initialization arguments can be passed via the :custom_args field.

%Membrane.Testing.Pipeline.Options {
    ...
    module: Your.Module,
    custom_args: [:your_custom_args]
}

Testing Source

Membrane.Testing.Source is an element that can output buffers from a data source passed as an option. It is especially useful when testing elements that network packets, e.g. RTP parser.

The data source can be either an enumerable:

%Membrane.Testing.Source{output: [0xA1, 0xB2, 0xC3, 0xD4]}

or a generator function:

initial_state = 1
generator_function = fn state, size ->
  # generate actions that source will return from handle_demand/4
  {actions, next_state}
end
%Membrane.Testing.Source{output: {initial_state, generator_function}}

Testing Sink

This element will notify the pipeline about buffers, caps and events it receives and by default make demands automatically. Alternatively, you can set an appropriate option and Membrane.Testing.Sink will no longer automatically make demands and will let you trigger demands by sending it a {:make_demand, size} message.

In conjunction with Membrane.Testing.Assertions it also allows you to assert on data it processed.

assert_sink_buffer(pipeline_pid, :sink ,%Membrane.Buffer{payload: 255})

Assertions

To make element integration test possible we introduced two groups of assertions.

First works in concjunction with Membrane.Testing.Pipeline:

assert_pipeline_notified(pipeline, element_name, notification_pattern, timeout)
assert_pipeline_playback_changed(pipeline, previous_state, current_state, timeout)
assert_pipeline_receive(pipeline, message_pattern, timeout)
refute_pipeline_receive(pipeline, message_pattern, timeout)
assert_start_of_stream(pipeline, element_name, pad, timeout)
assert_end_of_stream(pipeline, element_name, pad, timeout)

Second group works in conjunction with Membrane.Testing.Sink.

assert_sink_buffer(pipeline, sink_name, pattern, timeout)
assert_sink_caps(pipeline, element_name, caps_pattern, timeout)
refute_sink_caps(pipeline, element_name, caps_pattern, timeout)
refute_sink_buffer(pipeline, sink_name, pattern, timeout)
assert_sink_event(pipeline, sink_name, event, timeout)
refute_sink_event(pipeline, sink_name, event, timeout)

For details, please read docs.

Deprecations

Deprecation of Start and End of stream events in favor of custom callback

%Membrane.Event.StartOfStream{} and %Membrane.Event.EndOfStream{} will no longer be handled by handle_event/4 callback.

@impl true
def handle_event(pad, %Membrane.Event.StartOfStream{}, context, state) do
    # Do important work
    {:ok, state}
end
@impl true
def handle_event(pad, %Membrane.Event.EndOfStream{}, context, state) do
    # Do important work
    {:ok, state}
end

They will now be handled in their respective dedicated callbacks.

@impl true
def handle_start_of_stream(pad, context, state) do
  # Do important work
  {:ok, state}
end
@impl true
def handle_end_of_stream(pad, context, state) do
  # Do important work
  {:ok, state}
end

New in release v0.3.0

New way to define pads

To prepare for the next feature, we slightly changed the way pads are defined. Instead of defining them in groups with def_{in,out}put_pads/1 you should use def_{in,out}put_pad/2 for each pad.

So, this:

def_output_pads output: [mode: :push, caps: :any]

becomes this:

def_output_pad :output, mode: :push, caps: :any

The old way is now deprecated.
This small change results in nicer formatting of more complex definitions and prevents very deep list nesting, especially when defining...

Pad options

From now on, not only elements can have options - but pads as well. Options for a pad are defined similarly to the element's options:

  def_input_pad :input,
    availability: :on_request,
    demand_unit: :bytes,
    caps: :any,
    options: [
      mute: [
        type: :boolean,
        spec: boolean() | nil,
        default: nil,
        description: """
        Determines whether the pad will be muted from the start. If set to `nil` (default)
        the default from the element options will be used.
        """
      ]
    ]

The values for pad options are passed while linking pads (as keyword under :pad key). The former PullBuffer (now renamed to InputBuffer) options are passed under :buffer key:

  links = %{
      # ...
      {:file_src_b, :output} =>
        {:mixer, :input, pad: [mute: true], buffer: [preferred_size: 264_600]},
      # ...
  }

Push to pull connection

When linking push output with pull input pad, the 'toilet' mode is now enabled automatically. It can still be configured when linking pads

  links = %{
      # ...
      {:some_source, :output} =>
        {:some_sink, :input, buffer: [warn_size: 264_600, fail_size: 529_200]},
      # ...
  }

System time - but which system?

Membrane.Time.system_time/0 is now deprecated in favour of Membrane.Time.os_time/0 and Membrane.Time.vm_time/0 that make it clear from which system the time is taken.

Proxy functions for Membrane.Pipeline

You no longer have to use functions from Membrane.Pipeline module to start/play/stop the pipeline. Now every pipeline (that has use Membrane.Pipeline line) gets automatically generated proxy functions:
start/2, start_link/2, play/1, prepare/1, stop/1, stop_and_terminate/1

Documentation

The docs generation for elements have been revised. Now description of available options and pads (along with their options) are placed in moduledoc (like in Membrane.Testing.Sink)

Additionally, the docs for Membrane Core are properly grouped dramatically improving navigation

Changes

• updated Bunch to v1.0
• fixed leaking imports in Membrane.Log (#141)
• fixed some error handling related issues (#138)