Reader

The Reader monad now has more advanced error handling (which is also reflected in the code-gen that wraps the Reader monad).

This means Reader<Env, A>.Run(env) now returns ReaderResult<A> instead of TryOption<A>. I have also removed Filter and Where from Reader and instead you should use:

    from x in success
        ? Reader<Env, A>(successValue)
        : ReaderFail<Env, A>("Fail message")
   ...

Filter and Where would return a result in a Bottom state, which isn't ideal. Obviously if you need this functionality back then you can create the extension methods yourself to create a similar functionality:

public static Reader<Env, A> Where<Env, A>(this Reader<Env, A> ma, Func<A, bool> f) =>
    ma.Bind(a => f(a) ? Reader<Env, A>(a) : ReaderFail<Env, A>(BottomException.Default));

Fail states can be created using:

    ReaderFail<Env, A>(string message);
    ReaderFail<Env, A>(string message, Exception exception);
    ReaderFail<Env, A>(Exception exception);

ReaderResult

ReaderResult<A> has Match and IfNone (so replace previous usage of IfNoneOrFail with IfNone). It also has conversion methods: ToSeq(), ToList(), ToOption(), ToOptionUnsafe(), ToOptionAsync(), ToEither(), ToEither(Func<Error, L> f), ToEitherUnsafe(), ToEitherUnsafe(Func<Error, L> f), ToEitherAsync(), ToEitherAsync(Func<Error, L> f), ToTry(), ToTryAsync()

Error

To facilitate the better error handling I needed to add a new Error type. The chances of this clashing with user's code is large, so it has been put into a new namespace: LanguageExt.Common. It can hold a message, a status value int, and an Exception.

Result

It seems to me that Result, OptionalResult, and Error belong in the same namespace so Result and OptionalResult have been moved to LanguageExt.Common.

On the whole you shouldn't really see Error or Result, most of the time you'll just be doing member access - and so the need to include the LanguageExt.Common should be rare.

Unfortunately the previous fix to the HashMap and HashSet didn't catch all cases. This has now been updated and thoroughly tested.

If you're running any release of lang-ext since v3.3.0 (and using HashMap or HashSet) then it is advisable that you upgrade as soon as possible.

This is an important fix of an issue with the new HashMap and HashSet implementation (to patch any release since v3.3.0). There was a very sporadic issue with reading items from the map after removal of other items, which would seemingly create random/undefined behaviour, and so it is advised that you upgrade to this release asap.

Following on from the last two releases:

• Reader monad code-gen (improvements)
• Reader monad code generation

The LanguageExt.CodeGen system now supports wrapping the Reader/Writer/State (RWS) monad.

Usage of the RWS monad is complicated because it takes so many generic arguments:

    RWS<MonoidW, R, W, S, A> where MonoidW : struct, Monoid<W>

And so you can be forgiven for not giving it a try. For, the unitiated, the RWS monad is a super-powered monad that combines the features of the:

• Reader monad - in that is can take an environment. Think of this as read-only configuration state. This is useful to keep functions pure when working with 'global' state.
• Writer monad - which maintains a state that is a monoid. The most common monoid to use here would be a Seq<W>. The writer monad allows for easy logging of items without needing access to an external log. Again, keeping the computation pure. But can also be used to sum totals, concatenate strings, etc.
• State monad - which is like the Reader monad in that it manages a state object, but the state can be updated as the computation runs. The state is immutable and so operation is pure.

And so all of these features in a single monad provides a very powerful set of tools for writing pure functions. The unfortunate aspect of this is that the majority of the generic parameters are fixed for the lifetime of the computation but need to be provided far too regularly due to C#'s poor type-inference story: MonoidW, R, W. And even S could be fixed depending on its use-case.

And so this is where this release comes in, a code-gen system that wraps the RWS monad into something a bit more manageable.

    [RWS(WriterMonoid: typeof(MSeq<string>), Env: typeof(IO))]
    public partial struct Subsys<S, T> {}

The code above will create a wrapped RWS monad with an S state that can vary (by calling put(x)). Below, is a version that doesn't have a variable S generic parameter, and instead has a fixed state embedded in the wrapped RWS monad. So, put can only be called with another Person to update the state.

    [RWS(WriterMonoid: typeof(MSeq<string>), Env: typeof(IO), State: typeof(Person))]
    public partial struct Subsys<T> {}

The generated code for the second option looks like this:

    public partial struct Subsys<T>
    {
        readonly internal LanguageExt.RWS<LanguageExt.ClassInstances.MSeq<string>, TestBed.IO, LanguageExt.Seq<string>, TestBed.Person, T> __comp;
        internal Subsys(LanguageExt.RWS<LanguageExt.ClassInstances.MSeq<string>, TestBed.IO, LanguageExt.Seq<string>, TestBed.Person, T> comp) => __comp = comp;
        public static Subsys<T> Pure(T value) => new Subsys<T>((env, state) => (value, default, state, false));
        public static Subsys<T> Fail => new Subsys<T>((env, state) => (default, default, default, true));
        public Subsys<U> Map<U>(Func<T, U> f) => new Subsys<U>(__comp.Map(f));
        public Subsys<U> Select<U>(Func<T, U> f) => new Subsys<U>(__comp.Map(f));
        public Subsys<U> Bind<U>(Func<T, Subsys<U>> f) => new Subsys<U>(__comp.Bind(a => f(a).__comp));
        public Subsys<U> SelectMany<U>(Func<T, Subsys<U>> f) => new Subsys<U>(__comp.Bind(a => f(a).__comp));
        public Subsys<V> SelectMany<U, V>(Func<T, Subsys<U>> bind, Func<T, U, V> project) => new Subsys<V>(__comp.Bind(a => bind(a).__comp.Map(b => project(a, b))));
        public (TryOption<T> Value, LanguageExt.Seq<string> Output, TestBed.Person State) Run(TestBed.IO env, TestBed.Person state) => __comp.Run(env, state);
        public Subsys<T> Filter(Func<T, bool> f) => new Subsys<T>(__comp.Where(f));
        public Subsys<T> Where(Func<T, bool> f) => new Subsys<T>(__comp.Where(f));
        public Subsys<T> Do(Action<T> f) => new Subsys<T>(__comp.Do(f));
        public Subsys<T> Strict() => new Subsys<T>(__comp.Strict());
        public Seq<T> ToSeq(TestBed.IO env, TestBed.Person state) => __comp.ToSeq(env, state);
        public Subsys<LanguageExt.Unit> Iter(Action<T> f) => new Subsys<LanguageExt.Unit>(__comp.Iter(f));
        public Func<TestBed.IO, TestBed.Person, State> Fold<State>(State state, Func<State, T, State> f)
        {
            var self = this;
            return (env, s) => self.__comp.Fold(state, f).Run(env, s).Value.IfNoneOrFail(state);
        }

        public Func<TestBed.IO, TestBed.Person, bool> ForAll(Func<T, bool> f)
        {
            var self = this;
            return (env, s) => self.__comp.ForAll(f).Run(env, s).Value.IfNoneOrFail(false);
        }

        public Func<TestBed.IO, TestBed.Person, bool> Exists(Func<T, bool> f)
        {
            var self = this;
            return (env, s) => self.__comp.Exists(f).Run(env, s).Value.IfNoneOrFail(false);
        }

        public Subsys<T> Local(Func<TestBed.IO, TestBed.IO> f) => new Subsys<T>(LanguageExt.Prelude.local<LanguageExt.ClassInstances.MSeq<string>, TestBed.IO, LanguageExt.Seq<string>, TestBed.Person, T>(__comp, f));
        public Subsys<(T, U)> Listen<U>(Func<LanguageExt.Seq<string>, U> f) => new Subsys<(T, U)>(__comp.Listen(f));
        public Subsys<T> Censor(Func<LanguageExt.Seq<string>, LanguageExt.Seq<string>> f) => new Subsys<T>(__comp.Censor(f));
    }

    public static partial class Subsys
    {
        public static Subsys<T> Pure<T>(T value) => Subsys<T>.Pure(value);
        public static Subsys<T> Fail<T>() => Subsys<T>.Fail;
        public static Subsys<T> asks<T>(Func<TestBed.IO, T> f) => new Subsys<T>((env, state) => (f(env), default, state, false));
        public static Subsys<TestBed.IO> ask => new Subsys<TestBed.IO>((env, state) => (env, default, state, false));
        public static Subsys<TestBed.Person> get => new Subsys<TestBed.Person>((env, state) => (state, default, state, false));
        public static Subsys<T> gets<T>(Func<TestBed.Person, T> f) => new Subsys<T>((env, state) => (f(state), default, state, false));
        public static Subsys<LanguageExt.Unit> put(TestBed.Person value) => new Subsys<LanguageExt.Unit>((env, state) => (default, default, value, false));
        public static Subsys<LanguageExt.Unit> modify<T>(Func<TestBed.Person, TestBed.Person> f) => new Subsys<LanguageExt.Unit>((env, state) => (default, default, f(state), false));
        public static Subsys<T> local<T>(Subsys<T> ma, Func<TestBed.IO, TestBed.IO> f) => ma.Local(f);
        public static Subsys<T> Pass<T>(this Subsys<(T, Func<LanguageExt.Seq<string>, LanguageExt.Seq<string>>)> ma) => new Subsys<T>(ma.__comp.Pass());
        public static Subsys<T> pass<T>(Subsys<(T, Func<LanguageExt.Seq<string>, LanguageExt.Seq<string>>)> ma) => new Subsys<T>(ma.__comp.Pass());
        public static Subsys<(T, U)> listen<T, U>(Subsys<T> ma, Func<LanguageExt.Seq<string>, U> f) => ma.Listen(f);
        public static Subsys<T> censor<T>(Subsys<T> ma, Func<LanguageExt.Seq<string>, LanguageExt.Seq<string>> f) => ma.Censor(f);
        public static Subsys<LanguageExt.Unit> tell(LanguageExt.Seq<string> what) => new Subsys<LanguageExt.Unit>(tell<LanguageExt.ClassInstances.MSeq<string>, TestBed.IO, LanguageExt.Seq<string>, TestBed.Person, LanguageExt.Unit>(what));
        public static Subsys<LanguageExt.Seq<string>> ReadAllLines(string fileName) => ask.Map(__env => __env.ReadAllLines(fileName));
        public static Subsys<LanguageExt.Unit> WriteAllLines(string fileName, LanguageExt.Seq<string> lines) => ask.Map(__env => __env.WriteAllLines(fileName, lines));
        public static Subsys<int> Zero => ask.Map(__env => __env.Zero);
        public static Subsys<string> Name => get.Map(__env => __env.Name);
        public static Subsys<string> Surname => get.Map(__env => __env.Surname);
    }

You'll notice toward the end that the members of IO and Person are available to use directly.

• ReadAllLines method from IO (Reader environment)
• WriteAllLines method from IO (Reader environment)
• Zero property from IO (Reader environment)
• Name property from Person (State value)
• Surname property from Person (State value)

So, you can write:

    using Subsys;

    var fullName = from n in Name
                   from s in Surname
                   from _ in WriteAllLines(Seq(n, s))
                   select $"{n} {s}";

It's possible to pick'n'choose which features of the RWS you want to use by putting in Unit or MUnit for the types. This is a State and Reader monad with no useful Writer functionality:

    [RWS(WriterMonoid: typeof(MUnit), Env: typeof(IO), State: typeof(Person))]
    public partial struct Subsys<T> {}

As, with the Reader code-gen, it's possible to modify the constructor and failure functions:

    [RWS(WriterMonoid: typeof(MSeq<string>), Env: typeof(IO), State: typeof(Person), 
     Constructor: "Pure", Fail: "Error")]
    public partial struct Subsys<T> {}

The usual functions for RWS are available for access to the underlying state:

from s in get                 // to get the state-monad state
from _ in put(s)              // to put an updated state back into the state-monad
from e in ask                 // to get the reader-monad environment
from e in asks(io => ...)     // easy way to wrap some non-monadic functionality into the monad
from _ in tell(Seq1("Hello")) // to write to the writer-monad when the monoid is a `MSeq<string>`
from _ in modify(s => s)      // to modify the state inline
from r in local(ma, e => e)   // to provide an alternative environment for the `ma` monad to run in
                              // this could be considered running in a new scope for the duration 
                              // of the ma operation

The last release instroduced the new monad builder feature that wraps a Reader<Env, A> monad into a new monad with the Env hidden inside to make it easier to use.

This release improves on that feature by extending the static class that's generated with all functions, fields, and properties that are in the Env type.

And so, if we have an Env type like so:

    public interface IOEnv
    {
        Seq<string> ReadAllLines(string path);
        Unit WriteAllLines(string path, Seq<string> lines);
    }

And wrap it in a new monad called IO:

    [Reader(typeof(IOEnv))]
    public partial struct IO<A> {}

Then we can use the methods within the IOEnv type like so:

    var comp = from ls in IO.ReadAllLines("c:/test.txt")
               from __ in IO.WriteAllLines("c:/test-copy.txt", ls)
               select ls.Count;

Which simplifies the usage of the Reader monad even more.

NOTE: There is no requirement to use an interface for the Env, it can be any type.

It is also possible to specify the name of the constructor and failure functions: Return and Fail.

    [Reader(Env: typeof(IOEnv), Constructor: "LiftIO", Fail: "FailIO")]
    public partial struct IO<A> { }

This makes it much easier to use the static class as a namespace:

    using static IO;

    var comp = from ls in ReadAllLines("c:/test.txt")
               from __ in WriteAllLines("c:/test-copy.txt", ls)
               from x  in LiftIO(100)
               from y  in LiftIO(200)
               select x * y;

The generated code for the above example looks like this:

public partial struct IO<A>
{
    readonly LanguageExt.Reader<TestBed.IOEnv, A> __comp;
    internal IO(LanguageExt.Reader<TestBed.IOEnv, A> comp) => __comp = comp;
    public static IO<A> LiftIO(A value) => new IO<A>(env => (value, false));
    public static IO<A> FailIO => new IO<A>(env => (default, true));
    public IO<B> Map<B>(Func<A, B> f) => new IO<B>(__comp.Map(f));
    public IO<B> Select<B>(Func<A, B> f) => new IO<B>(__comp.Map(f));
    public IO<B> Bind<B>(Func<A, IO<B>> f) => new IO<B>(__comp.Bind(a => f(a).__comp));
    public IO<B> SelectMany<B>(Func<A, IO<B>> f) => new IO<B>(__comp.Bind(a => f(a).__comp));
    public IO<C> SelectMany<B, C>(Func<A, IO<B>> bind, Func<A, B, C> project) => new IO<C>(__comp.Bind(a => bind(a).__comp.Map(b => project(a, b))));
    public TryOption<A> Run(TestBed.IOEnv env) => __comp.Run(env);
    public IO<A> Filter(Func<A, bool> f) => new IO<A>(__comp.Where(f));
    public IO<A> Where(Func<A, bool> f) => new IO<A>(__comp.Where(f));
    public IO<A> Do(Action<A> f) => new IO<A>(__comp.Do(f));
    public IO<A> Strict() => new IO<A>(__comp.Strict());
    public Seq<A> ToSeq(TestBed.IOEnv env) => __comp.ToSeq(env);
    public IO<LanguageExt.Unit> Iter(Action<A> f) => new IO<LanguageExt.Unit>(__comp.Iter(f));
    public Func<TestBed.IOEnv, S> Fold<S>(S state, Func<S, A, S> f)
    {
        var self = this;
        return env => self.__comp.Fold(state, f).Run(env).IfNoneOrFail(state);
    }

    public Func<TestBed.IOEnv, bool> ForAll<S>(S state, Func<A, bool> f)
    {
        var self = this;
        return env => self.__comp.ForAll(f).Run(env).IfNoneOrFail(false);
    }

    public Func<TestBed.IOEnv, bool> Exists<S>(S state, Func<A, bool> f)
    {
        var self = this;
        return env => self.__comp.Exists(f).Run(env).IfNoneOrFail(false);
    }
}

public static partial class IO
{
    public static IO<A> LiftIO<A>(A value) => IO<A>.LiftIO(value);
    public static IO<A> FailIO<A>() => IO<A>.FailIO;
    public static IO<A> asks<A>(Func<TestBed.IOEnv, A> f) => new IO<A>(env => (f(env), false));
    public static readonly IO<TestBed.IOEnv> ask = new IO<TestBed.IOEnv>(env => (env, false));
    public static IO<LanguageExt.Seq<string>> ReadAllLines(string path) => ask.Map(__env => __env.ReadAllLines(path));
    public static IO<LanguageExt.Unit> WriteAllLines(string path, LanguageExt.Seq<string> lines) => ask.Map(__env => __env.WriteAllLines(path, lines));
}

A common use for the Reader monad is to pass through a static environment. This can often be configuration, but it could also be a collection of functions for doing dependency injection (doing it well, rather than the OO way).

For example:

    public interface IO
    {
        Seq<string> ReadAllLines(string path);
        Unit WriteAllLines(string path, Seq<string> lines);
    }

    public class RealIO : IO
    {
        public Seq<string> ReadAllLines(string path) => File.ReadAllLines(path).ToSeq();
        public Unit WriteAllLines(string path, Seq<string> lines)
        {
            File.WriteAllLines(path, lines);
            return unit;
        }
    }

This can then be used in a Reader computation:

var comp = from io in ask<IO>()
           let ls = io.ReadAllLines("c:/test.txt")
           let _  = io.WriteAllLines("c:/test-copy.txt", ls)
           select ls.Count;

Then the comp can be run with a real IO environment or a mocked one:

    comp.Run(new RealIO());

However, carrying around the, non-changing, generic environment argument has a cognitive overhead and causes lots of extra typing.

And so now it's possible to use the LanguageExt.CodeGen to wrap up the Reader<Env, A> into a simpler monad. i.e.

    [Reader(typeof(IO))]
    public partial struct Subsystem<A>
    {
    }

NOTE: For now the new monadic type must be a struct

NOTE ALSO: If you use multiple generic parameters then the last one will be the bound value type

When providing the [Reader...] attribute with the type of the environment parameter, the code-gen will build:

public partial struct Subsystem<A>
{
    readonly LanguageExt.Reader<TestBed.IO, A> __comp;
    internal Subsystem(LanguageExt.Reader<TestBed.IO, A> comp) => __comp = comp;
    public static Subsystem<A> Return(A value) => new Subsystem<A>(env => (value, false));
    public static Subsystem<A> Fail => new Subsystem<A>(env => (default, true));
    public Subsystem<B> Map<B>(Func<A, B> f) => new Subsystem<B>(__comp.Map(f));
    public Subsystem<B> Select<B>(Func<A, B> f) => new Subsystem<B>(__comp.Map(f));
    public Subsystem<B> SelectMany<B>(Func<A, Subsystem<B>> f) => new Subsystem<B>(__comp.Bind(a => f(a).__comp));
    public Subsystem<C> SelectMany<B, C>(Func<A, Subsystem<B>> bind, Func<A, B, C> project) => new Subsystem<C>(__comp.Bind(a => bind(a).__comp.Map(b => project(a, b))));
    public Subsystem<TestBed.IO> Ask => new Subsystem<TestBed.IO>(LanguageExt.Prelude.ask<TestBed.IO>());
    public TryOption<A> Run(TestBed.IO env) => __comp.Run(env);
    public Subsystem<A> Where(Func<A, bool> f) => new Subsystem<A>(__comp.Where(f));
    public Subsystem<A> Filter(Func<A, bool> f) => new Subsystem<A>(__comp.Filter(f));
    public Subsystem<A> Do(Action<A> f) => new Subsystem<A>(__comp.Do(f));
    public Subsystem<A> Strict() => new Subsystem<A>(__comp.Strict());
    public Seq<A> ToSeq(TestBed.IO env) => __comp.ToSeq(env);
    public Subsystem<LanguageExt.Unit> Iter(Action<A> f) => new Subsystem<LanguageExt.Unit>(__comp.Iter(f));
    public Func<TestBed.IO, S> Fold<S>(S state, Func<S, A, S> f)
    {
        var self = this;
        return env => self.__comp.Fold(state, f).Run(env).IfNoneOrFail(state);
    }

    public Func<TestBed.IO, bool> ForAll<S>(S state, Func<A, bool> f)
    {
        var self = this;
        return env => self.__comp.ForAll(f).Run(env).IfNoneOrFail(false);
    }

    public Func<TestBed.IO, bool> Exists<S>(S state, Func<A, bool> f)
    {
        var self = this;
        return env => self.__comp.Exists(f).Run(env).IfNoneOrFail(false);
    }
}

public static partial class Subsystem
{
    public static Subsystem<A> Return<A>(A value) => Subsystem<A>.Return(value);
    public static Subsystem<A> Fail<A>() => Subsystem<A>.Fail;
    public static Subsystem<A> asks<A>(Func<TestBed.IO, A> f) => new Subsystem<A>(env => (f(env), false));
    public static readonly Subsystem<TestBed.IO> ask = new Subsystem<TestBed.IO>(env => (env, false));
}

This then allows for the example code above to become:

var comp = from io in Subsystem.ask
           let ls = io.ReadAllLines("c:/test.txt")
           let _  = io.WriteAllLines("c:/test-copy.txt", ls)
           select ls.Count;

As you can see it has provided wrappers for all of the useful functions of the Reader monad into two partial types that can then easily be extended. This makes it much easier to work with the Reader: the generated monad will behave almost exactly like any of the other simpler monads like Option, but will have access to a 'hidden' environment.

This is the first step to simplifying the use of Reader<Env, A>, Writer<MonoidW, W, A>, State<S, A>, and RWS<MonoidW, R, W, S, A>. The future versions will also generate the Monad class-instance, as well as generate the transformer stack for easy nesting of the new monad.

Language-ext has been in beta for over a month, this is the first full release since. It features:

• Performance improvements for Seq<A>, HashMap<A>, and HashSet<A>
• Improved perfomance and reduced memory usage by Option<A>, OptionUnsafe<A>.
• Software Transcational Memory using Multi-Version Concurrency Control
  • Atom<A>
  • Ref<A>

Breaking changes:

• Seq<A> is now a struct and so can't be null. This mostly isn't a problem for code that needs to check for null to initialise the collection as empty, but if it does something else then that will fail. So, look for uses of Seq<A> and just validate that you'll be ok.

One aspect of using immutable data-types like Map, Seq, HashSet, etc. is that in most applications, at some point, you're likely to have some shared reference to one and need to mutate that shared reference. This often requires using synchronisation primitives like lock (which are not composable and are prone to error).

Atom

With a nod to the atom type in Clojure language-ext now has two new types:

• Atom<A>
• Atom<M, A>

These types all wrap a value of A and provides a method: Swap (and Prelude function swap) for atomically mutating the wrapped value without locking.

var atom = Atom(Set("A", "B", "C"));

atom.Swap(old => old.Add("D"));
atom.Swap(old => old.Add("E"));
atom.Swap(old => old.Add("F"));

Debug.Assert(atom == Set("A", "B", "C", "D", "E", "F"));

Atomic update

One thing that must be noted is that if another thread mutates the atom whilst you're running Swap then your update will rollback. Swap will then re-run the provided lambda function with the newly updated value (from the other thread) so that it can get a valid updated value to apply. This means that you must be careful to not have side-effects in the Swap function, or at the very least it needs to be reliably repeatable.

Validation

The Atom and AtomRef constructors can take a Func<A, bool> validation lambda. This is run against the initial value and all subsequent values before being swapped. If the validation lambda returns false for the proposed value then false is returned from Swap and no update takes place.

Events

The Atom and AtomRef types all have a Change event:

    public event AtomChangedEvent<A> Change;

It is fired after each successful atomic update to the wrapped value. If you're using the LanguageExt.Rx extensions then you can also consume the atom.OnChange() observable.

Metadata

The two types, with an M generic argument, take an additional meta-data argument on construction which can be used to pass through an environment or some sort of context for the Swap functions:

var env = new Env();

var atom = Atom(env, Set("A", "B", "C"));

atom.Swap((nenv, old) => old.Add("D"));

Additional arguments

There are also other variants of Swap that can take up to two additional arguments and pass them through to the lambda:

var atom = Atom(Set(1, 2, 3));

atom.Swap(4, 5, (x, y, old) => old.Add(x).Add(y));

Debug.Assert(atom == Set(1, 2, 3, 4, 5));

Accessing the value

The wrapped value can be accessed by calling atom.Value or using the implicit operator conversion to A.

Pool

The updates to the object pooling system also improves the performance of the Lst, Set, and Map enumerators.

Conclusion

I hope the Atom types finds some use, I know I've bumped up against this issue many times in the past and have either ended up manually building synchronisation primitives or fallen back to using the ugly ConcurrentDictionary or similar. I will perhaps take a look at the ref system in Clojure too - which is a mechanism for atomic updates of multiple items (STM essentially).

Atom source code

HashSet has been switched to use the same TrieMap that HashMap uses. This will give a similar performance boost to HashSet.

There have been some more tweaks to TrieMap performance too. Which I'm keeping track of here.

More to come!

HashMap<K, V> and HashMap<EqK, K, V> are now about 4 times faster. The underlying implemenation has now switched from being a Map<int, Lst<(K, V)> to a Compressed Hash Array Map Trie (CHAMP).

This still has some more performance to squeeze out of it as it's a relatively naive implementation at the moment, so I won't post up the figures just yet.

Included in this release are performance improvements for Option<A>, OptionUnsafe<A>, and Seq<A>. Seq<A> has now also become a struct to stop the usage of null but also to facilitate the new performance improvements. ISeq<A> will eventually be deprecated and so there are now [Obsolete] warnings - but I won't kill it for quite a long time, I just want to discourage its usage.

I'm doing this as a beta release because the changes to Seq<A> are enormous - well, a complete re-write. It is now entirely lock free and internally has three possible states: Lazy, Strict, and Empty - which allows for improved performance for the happy paths.

I will keep language-ext in beta for a while as I go through the library optimising what I believe are the most used features. I am doing this now because I feel the library is settling down now - as in there's less churn of features and interfaces - and so now is a good time to start squeezing the most performance out of it.

Mostly this will involve trying to reduce memory allocations where possible as well as inlining operations that make unnecessary calls (which were there to keep everything super generic under the hood). This will make long term maintenance a bit harder, which is why it needed to wait until now.

Performance timings

Below are some tests I've used to tweak the performance, the timings are all relative from the old Seq to the new, so my machine's spec shouldn't matter.

Streaming a lazy sequence

This performance test wraps a lazy enumerable in a Seq and then streams them in one-by-one via the foreach loop and saves the value in a results array.

var seq = Seq(Range(0, count));
foreach(var item in seq)
{
    results[j] = item;
    j++;
}

So a performance improvement of 2.55 times. Evaluating an IEnumerable<T> lazy sequence is about 29 ns / op which whilst faster than Seq<A> doesn't have the memoisation that Seq has. (or any of the other cool features, for that matter)

Streaming a strict sequence

A lot of the time I find I'm working with strict sequences - i.e. non-lazy. This is the happy path for sequences and a lot of optimisations can happen if the code knows the sequence isn't lazy

var seq = Seq(Range(0, count));
seq = seq.Strict();

foreach (var item in seq)
{
    results[j] = item;
    j++;
}

A performance improvement of ~32.3%. This isn't quite as significant as the lazy stream improvements, but interestingly this is now faster than List<T> from the BCL, which measures around 12.5 ns / op.

Adding an item to the end of a Seq

Essentially calling this, many times:

seq = seq.Add(value);

Around a 33% improvement. This isn't quite as fast as the 10.9 ns / op of the BCL's List<T>, I will work on this some more. But this is an immutable, thread-safe, data structure - which the BCL List<T> definitely isn't.

Adding an item to the beginning of a Seq

Essentially calling this, many times:

seq = value.Cons(seq);

Twice as fast. This isn't far off List<T>.Add - and so that's why I think I can make Seq<A>.Add gain a bit more speed. Note: the BCL's List<T>.Insert(0, x) which is the equivalent to Seq<A>.Cons has terrible performance at 10310 ns / op. So that's one to look out for!

Other Seq<A> functions

Most Seq<A> functions will either evaluate a lazy stream, a strict stream, or Add or Cons items. And so nearly all Seq<A> related functionality will gain due to these changes.

Option<A> and OptionUnsafe<A>

I haven't done any performance timings for these types, as most improvements are to reduce unnecessary allocations. I have removed the support for lazy options - if anyone misses them then I'll probably create an OptionLazy<A> type. I'd rather not, but I felt that because Option<A> is probably the most used type in lang-ext then it should always be on the happy path. This has reduced a lot of unnecessary branching and has allowed the internals of Option<A> and OptionUsafe<A> to be simplified.

Feedback please

Because these are some big changes, if you have an app with lots of unit tests it'd be great if you could verify that they all still run with these updates. I have obviously made sure all of the existing unit tests pass and have built a number of test harnesses to check that everything still works and to verify the thread-safeness of Seq, but changes like these have a habit of kicking you when you least expect it. So, any help will be gratefully received.

The LanguageExt.CodeGen tool has been updated to spot PascalCase field names that will become C# identifiers when made into camelCase for lenses (and to prepend an @ to the name). So, names like Class, Default, Event, etc. don't cause compilation problems.

NOTE: The other LanguageExt.* nu-get packages haven't been deployed. I'm currently working on some optimisations that will need some more testing before release.

The [With] and [WithLens] code-generator has been further improved:

• Supports generic types
• Supports types with constraints
• Can ascertain the type for With parameters and can therefore ascertain whether it's a value-type or reference-type. This allows for the parameters to use Nullable<A> for value-types, therefore massively helping with implicit type-conversion.
• Removed WithOpt which was playing the roll of Nullable before the type resolution that's now been added.

NOTE: Pre-C# 8 - you must put a constraint on your generic arguments (where A : struct or whera A : class) to allow for the null-coalescing to work correctly. The null-coalesce operator has been improved in the latest C# and so this requirement isn't needed.

A stupidity bug unfortunately slipped through the net, which has now been fixed with the WithOpt<A> type which was built to provide sensible defaults for the [With] and [WithLens] code-gen. So, if you're using the code-gen features, you're gonna want this release.

The Do operation for running side-effects on monadic computations was running the computations for lazy monads (Try, TryOption, TryAsync, TryOptionAsync, Reader, Writer, State, RWS) twice. This has now been fixed.

Strict hadn't been implemented for all lazy monadic types. This has now also been rectified.

The LanguageExt.CodeGen library has been updated to v3.1.20 - this will now deal with Option types for fields correctly. Previously, None wasn't handled as a value, but as an absence of a value. So if you use [With] or [WithLens] and you have fields in your record type that use Option<...> then you'll need this update.

The last release (v3.1.16) had a problem with static field construction in Lst<A> and Arr<A> which caused a TypeLoadException.

This release patches that issue and an issue causing a stack-overflow with the new EnumerableOptimal.ConcatFast utility.

Lenses allow for bi-directional transformation of immutable value types. This is particularly needed when immutable types are composed and transformation results in many nested With functions or getters and setters.

Transformation of immutable types

If you're writing functional code you should treat your types as values. Which means they should be immutable. One common way to do this is to use readonly fields and provide a With function for mutation. i.e.

public class A
{
    public readonly X X;
    public readonly Y Y;

    public A(X x, Y y)
    {
        X = x;
        Y = y;
    }

    public A With(X X = null, Y Y = null) =>
        new A(
            X ?? this.X,
            Y ?? this.Y
        );
}

Then transformation can be achieved by using the named arguments feature of C# thus:

val = val.With(X: x);

val = val.With(Y: y);

val = val.With(X: x, Y: y);

[With]

It can be quite tedious to write the With function however. And so, if you include the LanguageExt.CodeGen nu-get package in your solution you gain the ability to use the [With] attribtue on a type. This will build the With method for you.

NOTE: The LanguageExt.CodeGen package and its dependencies will not be included in your final build - it is purely there to generate the code.

You must however:

• Make the class partial
• Have a constructor that takes the fields in the order they are in the type
• The names of the arguments should be the same as the field, but with the first character lower-case

i.e.

[With]
public partial class A
{
    public readonly X X;
    public readonly Y Y;

    public A(X x, Y y)
    {
        X = x;
        Y = y;
    }
}

Transformation of nested immutable types with Lenses

One of the problems with immutable types is trying to transform something nested deep in several data structures. This often requires a lot of nested With methods, which are not very pretty or easy to use.

Enter the Lens<A, B> type.

Lenses encapsulate the getter and setter of a field in an immutable data structure and are composable:

[With]
public partial class Person
{
    public readonly string Name;
    public readonly string Surname;

    public Person(string name, string surname)
    {
        Name = name;
        Surname = surname;
    }

    public static Lens<Person, string> name =>
        Lens<Person, string>.New(
            Get: p => p.Name,
            Set: x => p => p.With(Name: x));

    public static Lens<Person, string> surname =>
        Lens<Person, string>.New(
            Get: p => p.Surname,
            Set: x => p => p.With(Surname: x));
}

This allows direct transformation of the value:

var person = new Person("Joe", "Bloggs");

var name = Person.name.Get(person);
var person2 = Person.name.Set(name + "l", person);  // Joel Bloggs

This can also be achieved using the Update function:

var person = new Person("Joe", "Bloggs");

var person2 = Person.name.Update(name => name + "l", person);  // Joel Bloggs

The power of lenses really becomes apparent when using nested immutable types, because lenses can be composed. So, let's first create a Role type which will be used with the Person type to represent an employee's job title and salary:

[With]
public partial class Role
{
    public readonly string Title;
    public readonly int Salary;

    public Role(string title, int salary)
    {
        Title = title;
        Salary = salary;
    }

    public static Lens<Role, string> title =>
        Lens<Role, string>.New(
            Get: p => p.Title,
            Set: x => p => p.With(Title: x));

    public static Lens<Role, int> salary =>
        Lens<Role, int>.New(
            Get: p => p.Salary,
            Set: x => p => p.With(Salary: x));
}

[With]
public partial class Person
{
    public readonly string Name;
    public readonly string Surname;
    public readonly Role Role;

    public Person(string name, string surname, Role role)
    {
        Name = name;
        Surname = surname;
        Role = role;
    }

    public static Lens<Person, string> name =>
        Lens<Person, string>.New(
            Get: p => p.Name,
            Set: x => p => p.With(Name: x));

    public static Lens<Person, string> surname =>
        Lens<Person, string>.New(
            Get: p => p.Surname,
            Set: x => p => p.With(Surname: x));

    public static Lens<Person, Role> role =>
        Lens<Person, Role>.New(
            Get: p => p.Role,
            Set: x => p => p.With(Role: x));
}

We can now compose the lenses within the types to access the nested fields:

var cto = new Person("Joe", "Bloggs", new Role("CTO", 150000));

var personSalary = lens(Person.role, Role.salary);

var cto2 = personSalary.Set(170000, cto);

[WithLens]

Typing the lens fields out every time is even more tedious than writing the With function, and so there is code generation for that too: using the [WithLens] attribute. Next, we'll use some of the built-in lenses in the Map type to access and mutate a Appt type within a map:

[WithLens]
public partial class Person : Record<Person>
{
    public readonly string Name;
    public readonly string Surname;
    public readonly Map<int, Appt> Appts;

    public Person(string name, string surname, Map<int, Appt> appts)
    {
        Name = name;
        Surname = surname;
        Appts = appts;
    }
}

[WithLens]
public partial class Appt : Record<Appt>
{
    public readonly int Id;
    public readonly DateTime StartDate;
    public readonly ApptState State;

    public Appt(int id, DateTime startDate, ApptState state)
    {
        Id = id;
        StartDate = startDate;
        State = state;
    }
}

public enum ApptState
{
    NotArrived,
    Arrived,
    DNA,
    Cancelled
}

So, here we have a Person with a map of Appt types. And we want to update an appointment state to be Arrived:

// Generate a Person with three Appts in a Map
var person = new Person("Paul", "Louth", Map(
    (1, new Appt(1, DateTime.Parse("1/1/2010"), ApptState.NotArrived)),
    (2, new Appt(2, DateTime.Parse("2/1/2010"), ApptState.NotArrived)),
    (3, new Appt(3, DateTime.Parse("3/1/2010"), ApptState.NotArrived))));

// Local function for composing a new lens from 3 other lenses
Lens<Person, ApptState> setState(int id) => 
    lens(Person.appts, Map<int, Appt>.item(id), Appt.state);

// Transform
var person2 = setState(2).Set(ApptState.Arrived, person);

Notice the local-function which takes an ID and uses that with the item lens in the Map type to mutate an Appt. Very powerful stuff.

There are a number of useful lenses in the collection types that can do common things like mutate by index, head, tail, last, etc.

Everything else

• Added ma.Flatten() and flatten(ma) monadic join for all monadic types
• Added Indexable type-class and class-instances for all indexable types
• Performance improvements for enumerable bind and enumerable concat (which should also protect against the Stack Overflow on .NET 4.6)
• Fix for BindT being eager with IEnumerable
• Deprecated some duplicate extensions for Task
• [Pure] attribute should now propagate to the assemblies

All on nu-get now.

• Bug fix517 throw on option map to null
• added .Take(1) optimization to HeadOrNone
• Correct null checks
• Correct spelling mistakes
• Record fix
• Add ordering type class to NewType
• Add Regex match string predicate
• Improve FoldValidation
• Bugfix for Try / TryOption (mostly bottom state)
• Just some unit tests and comment fixes

Thanks to all those submitting pull-requests and reporting issues 👍

There was an intermittent problem with lazy Seq which would occasionally cause iteration to skip an item in the sequence. This could happen if two iterators were consuming the sequence separately (not a threading issue, but a state issue). Thanks to Tom @colethecoder for fixing that issue.

Anybody using Language-Ext 3.1.* will want to upgrade to avoid this issue.

There is a new Do function on every monadic and collection type. It's the same as Iter but returns the original container type. This allows for fluent side-effecting code.

On nu-get now.