Raf's laboratory Raffaele Rialdi personal website

It's been a long time since I last posted here and many things happened in the past.

Anyway, nothing changed in my eternal reasearch in improving the software quality, experimenting with new technologies, sharing with the communities I love.

Many years ago, I started travelling across the world for conferences, from USA to Siberia passing by the lovely Cluj Napoca in Romania and many other beautiful cities. But of course, on 2020 everything suddenly stopped because of the covid, where five of the conferences I was speaker on in spring had to be cancelled. But this is not a report of my past travels and I want to focus on the software programming topics I love most.

So, let me restart, I have a long list of things I always wanted to write on, and it's time to make them happen.

Over these years, the .NET panorama has dramatically changed with the advent of .NET Core, but more recently a number of other technologies are coming and during conferences I am often asked how do they fit in the panorama. In order to understand how all the pieces are going to compose the puzzle, I will digress on the metadata and a bit of history.

I love .NET is metadata, let me stress why. The C language, which is still widely used in low-level programming (kernel mode, embedded development, etc.) still plays a central role in modern application development because of its binary contract. While it is very basic and simple, when I export a function in C with 'cdecl' or 'stdcall' I can use the binary entry-point of that function from any other language without having to recompile. This is a very primitive form of metadata.

The C++ language brought OOP (and more recently function) abstractions, but the ISO committee never defined a binary standard which is (IMO) the main reason for C++ suffering in comparison to other languages. This basically means that you have to recompile every library exporting C++ types (classes) because the binary format may change for every compiler brand and version.

As a side note, few years ago, the Microsoft C++ team made a gigantic restructure of the Runtime Libraries in order to minimize the breaking changes over time. They can now guarantee a bit more of compatibility between different versions of their own compiler/standard libraries.

Since there was no binary contract in C++ but still people wanted to deal with complex types and not just mere C functions, a new metadata-oriented language was invented: IDL (Interface Definition Language). IDL is a standard specification providing a way to describe what happens when you need to communicate between two different languages. It essentially defines an extensible type system, how the vtable of a binary (compiled) complex type and the input/output direction of the parameters, which is vital in out-of-process scenario to avoid useless memory copies.

Microsoft built a glorious infrastructure over IDL with the aim to standardize the rules to invoke components in in-process, out-of-process and out-of-machine scenarios: The Component Object Model (COM). You may have hated or loved it (I loved it, and you may recall Don Box motto "COM is Love") because it solved huge problems in terms of code reuse. COM is still heavily used in Windows and in .NET as well, even in .NET Core when running on Linux (I will dig on this in another post). But COM was not the sole implementation, in the *nix world it came Corba which didn't have the expected success due to the different (and incompatible) implementations available. Still Corba is the perfect demonstration that the IDL metadata paradigm was a good and practical solution.

The main problems behind COM where substantially two: the awkward type system (BSTR, VARIANT, …) and the difficulties in managing the concept of 'Apartments' (threading model) which is basically the reason why VB and VBScript never exposed multithreading. A third problem, registering components in the Windows registry was solved more recently.

Not surprisingly .NET was the successor of COM (its internal name was COM3, if I recall correctly the story from one of their creators), but I consider .NET a "fork" (from the design perspective) more than a successor.

When .NET was in beta at the end of the 90's I was in Davos at the WinSummit conference and listening to Jeffrey Richter about this new project called .NET. I was shocked to hear about IDL going away, scared about missing deterministic destructors and loved since the very beginning about the "metadata" story.

Metadata is life for any software system, and .NET defined the specification that later were approved as the famous "ECMA-335" standard. The same spec was adopted in WinRT and now still at the heart of .NET Core and before . Let me open a small parenthesis on WinRT. In 2011 I was in Los Angeles, enthusiastically listening to the WinRT (Windows Runtime) launch. I loved it since the beginning because I knew the details of the story I just wrote here.

WinRT is essentially COM using ECMA-335 metadata, a simplified threading model using the "free-threaded marshaler" by default and a revamped type system which basically the same of the one used in .NET. During the conference I told to myself: "Bingo!". They will now write all the Windows API as WinRT and this happened! No more PInvokes for my own components … this was technically possible but was inhibited in WinRT. I will be able to use WinRT API from the OS from all my existing Apps … but this happened only very recently and we will have to wait for .NET 5 before seeing the true power of that. They will make me write custom components as WinRT and this will allow to mix languages in a super-simple way … but sadly this never happened and without technical reasons (the component factory disallowed the creation of those components when called from 'desktop' apps). Those limitations were there just because they hoped to 'force' the developers to adopt the 'Modern Apps' development, culminated in the UWP paradigm, which failed its goals. We know very well that those were bad mistakes and now Microsoft knows it. In a future post I will digress on how .NET 5 is going to start fixing that.

As UWP is not getting adoption, is WinRT dead? Not at all! The infrastructure is gold and works great, it just needed more wisdom. Many years ago, Kenny Kerr was a C++ MVP and we had the occasion to discuss many topics over the years. I remember when he asked me to test his private project called "cppwinrt". I was enthusiastic of his project and so I did, asking him to add more features like async support etc. Its goal was to create a C++ projection to nicely use WinRT components and there were multiple reasons for that. The first is that C++/CX (a C++ custom dialect) were not part of the standard and had poor performances. Another reason is that using components (WinRT and COM) never felt natural and required CreateInstance, QueryInterface, etc. Kenny's solution was to read winmd/idl metadata to generate the C++ source code where you could just constructors, operators, etc. as if the component was a local class. Later Kenny joined the Windows teams in Microsoft to continue evolving his project.

What comes after cppwinrt? It is Project Reunion, which is basically filling the gaps with WinRT. Project Reunion is willing to expose the OS features with the WinRT ABI (Abstract Binary Interface) providing a way for any language to interop with those components. The new WinUI initiative is an example of an important piece exposed with the WinRT contract. Of course, you need more language projections, for example there are (but aren't completed yet) is the Python projection and the C# projection which is part of the .NET 5 wave.

And remember that all of this is possible just because of metadata, which the soul of the software.