The 10th anniversary of SObjectizer-5!

The first commit of SObjectizer-5 project was made in Oct 2010. Soon after that SObjectizer-5 started working and we consider the end of 2010 as a SObjectizer-5's date of birth. And now, at the end of 2020 we can celebrate the 10th anniversary of SObjectizer-5.

• SObjectizer-5 origins and history
  • Years 2002-2006
  • 2010-2013
  • 2013-present days
    • so5extra should also be mentioned
• The failed attempt of making SObjectizer popular
  • Where and how SObjectizer was used in real-life?
• Why we worked on SObjectizer and why we still work on it?
• The current state and the future of the project
• Acknowledgements

Note. If you didn't hear about SObjectizer earlier then this overview can give you some insight about the project itself and niches where it was or can be used.

SObjectizer-5 origins and history

The story of SObjectizer began in 2002 (not speaking of more ancient SCADA Objectizer project in 1990s) and the whole history of that project can be split into several parts: 2002-2006 with the birth of SObjectizer-4 and its first appearance in the public, 2010-2013 with the birth and initial growth of SObjectizer-5 and from 2013 till now with SObjectizer-5 as an independent OpenSource project.

Years 2002-2006

The history of SObjectizer itself began in Apr 2002 when a very small framework for the simplification of the development of complex multithreaded applications in C++ was created inside JSC Intervale company. That framework was called SObjectizer-4 (or SO-4 for the short).

SO-4 was used as a base for several Intervale's business-critical projects in the early 2000s. Some of them were ported to Java, some of them were closed after years in production, some are still in use.

SObjectizer-4 evolved rapidly for two years. Somewhere at the end of 2004 or at the beginning of 2005 SO-4 was switched to a great ACE library instead of our own handmade code for work with threads, mutexes, sockets, and other OS-dependent stuff.

In 2005 and 2006 it became obvious that we hit some ceiling in the abilities to make SObjectizer-4 more powerful and helpful.

So the decision to open SObjectizer-4 was made and tarballs with SObjectizer's sources were published on SourceForge in 2006.

SO-4 didn't get serious attention. But after some discussions with people who showed an interest in SObjectizer, we decided that SObjectizer-4 in its form has no future. And we started to think about how to create a new and better version of SObjectizer.

And this took almost four years.

2010-2013

In 2010 a new C++team was formed in JSC Intervale and we started the development of SObjectizer-5 in the autumn of 2010. We reused the core principles of SObjectizer-4 but made several decisions quite differently.

For example, only return codes were used in SO-4 for error reporting. That led to several cases when important errors were ignored that were detected only in production. Because of that SO-5 uses exceptions.

A special DSL based on C-macros was used in SObjectizer-4 for a description of agents, their states, and messages. That DSL was completely removed. In SO-5 we don't need to use macros and don't need to do additional description of agents/messages/states.

SO-4 supported the building of distributed applications. That feature was completely removed from SO-5 due to several reasons (we won't speak about that here).

And last, but not least, we decided to target C++11. Despite the fact that we haven't C++11 conformant compilers at the time of the birth of SObjectizer-5. And that decision had its consequences...

Anyway, SObjectizer-5 was born in late 2010 and we started to use it for writing production code in 2011. AFAIK, some of the code written with SO-5 in those times is still in use.

SObjectizer-5 was created with the idea of being an OpenSource project from the very beginning. But it took several years to make a battle-tested version that we can show to the public. So the first public release of SObjectizer-5 was made only in the spring of 2013. Soon after that, the whole development of SO-5 migrated from JSC Intervale to SourceForge and SObjectizer became a fully independent project that is no more related to its mother company.

2013-present days

Since the mid of 2013 SObjectizer-5 has been developed by a small group of former colleagues named the SObjectizer Team.

In 2013 and 2014 a lot of work was done to polish SObjectizer's internals, translating comments in the code from Russian to English, writing docs and examples. SObjectizer went from v.5.1 to v.5.5 with several more or less breaking changes between them. On that way, we decided to part with ACE, and since v.5.5 SObjectizer doesn't rely on ACE at all (most of the stuff is now in C++ standard library, and timers we had implemented yourself).

After a release of v.5.5, we have taken care of keeping strict compatibility between the releases inside SObjectizer-5.5 and kept that care for more than five years. SObjectizer-5.5 gained a lot of new features between 2014 and 2019 years but in most cases, the switch to a new version required only recompilation with new SO-5 sources.

A significant shift was made in spring 2019 during the development of SObjectizer-5.6 branch. We decided not only to target C++17 instead of C++11, but also to renew some parts of SObjectizer-5. And it seems that SObjectizer-5.6 became more helpful and easy to use than SObjectizer-5.5 was.

Another compatibility break was made at the beginning of 2020 when SObjectizer-5.7 was released. There were only a few user-visible changes between v.5.7 and v.5.6, the most notable is the rename of the case_ function to receive_case. But we decided to bump the version number to reflect the fact that some fixes in source code can be necessary during the upgrade to v.5.7.

Shortly speaking since 2013 SObjectizer-5 went a long way from a very small library with a couple of examples and sparse docs to a project that has incorporated a lot of new features, received rather good documentation, dozens of examples, and hundreds of tests.

so5extra should also be mentioned

In 2017 we started the development of a companion project so5extra. That project began as an attempt to make some money selling commercial licenses for additional stuff, because of that the first versions of so5extra were distributed on a dual-licensing model.

But this idea didn't succeed and since v.1.4 so5extra is distributed under the same license as SObjectizer-5 itself -- BSD-3-Clause. So now we see so5extra as a part of the whole SObjectizer project. And if you miss something from the core SO-5 just look inside so5extra, maybe it already has what you need.

The failed attempt of making SObjectizer popular

It would be great if SObjectizer became a well-known project in the C++ community. But that didn't happen.

There are several reasons for this sad fact. And we'll mention only a few of them here.

We hosted the source code of SObjectizer on SourceForge for a long time. In a Subversion repository. At the time when SourceForge almost killed its reputation by adding additional software to releases of unmaintained projects...

Then we switched to the Mercurial repository on BitBucket. Just several months before the Atlassian decision to kill Mercurial support on BitBucket.

And only then we moved our main repo to GitHub. There were reasons why we stayed on Svn and SourceForge for several years, but that was a mistake in the end.

Maybe the support for CMake was added too late. We use our own build-system written in Ruby in 2004 and the first published versions of SObjectizer-5 contained only Ruby scripts for building SO-5, examples and tests. The official support for CMake was added only a few years later.

We started spreading information about SObjectizer too late. The first announcements in Russian were made in the mid of 2014. And we decided to leave the Russian segment of the Internet and share the info about SObjectizer in English only in 2016. But even after that, most of SObjectizer-related posts/articles/reports were given in Russian. Our posts and announcements in English on some resources (like Reddit and Hacker News) didn't get interest.

But the most important factor is that we are not good at PR at all. We are software engineers, not marketers. We solve technical problems, write, and document code. But it isn't enough to make a software project popular even if we believed that the project is useful and that was proved with time.

Perhaps, another reason is the drop in popularity of C++ in the last 15 years. Maybe, if we switched from C++ to Go or Rust, our project would have gained more attention, who knows? :)

Where and how SObjectizer was used in real-life?

And last but not least, another reason for our failed PR attempts is our inability to tell good success stories about the SObjectizer usage in real-life projects. It always was hard to speak about how and where SObjectizer was used. Because of the several factors.

First of all, we don't know about all the usages of SObjectizer. SObjectizer is free software that can be used even in proprietary projects. So some people just get SObjectizer and use it without providing any feedback for us. Sometimes we acquire some information about SObjectizer's usage (but without any details), sometimes we know nothing.

The second problem is permission to share information about the usage of SObjectizer in a particular project. We have received that permission very rarely, in most cases users of SObjectizer do not want to open implementation details of their projects (sometimes we understand the reasons, sometimes don't).

We apologize for the fact that the information provided looks so scarce and does not contain any details. Nevertheless, there are some examples of usage of SObjectizer:

• SMS/USSD aggregation gateway that handled more than 500M of messages per month;
• part of the system served online payments via ATMs of one of the biggest Russian banks;
• simulation modeling of economic processes (as part of Ph.D. research);
• distributed data acquisition and analytic system. Data collected on points distributed worldwide by the commands from the central node. MQTT was used as a transport for control and acquired data distribution;
• testing environment for checking real-time control system for railway equipment;
• automatic control system for theatre scenery. More details can be found here;
• components of a data management platform in an online advertising system.

All of those above are proprietary and closed source projects. But if someone wants to look at how more or less real-life code based on SObjectizer can look like then there is a small demo-project Shrimp we created right for that purpose.

Why we worked on SObjectizer and why we still work on it?

It's a simple and hard question at the same time. There are various reasons and a couple of them will be discussed here.

One of the reasons is our desire to share our experience with SObjectizer. SObjectizer helped us a lot in the past, it helps us in the present. We think it's a good tool and because of that it can be useful for someone else too.

But as a tool SObjectizer can be used incorrectly. It's not a silver bullet and never pretended to be. We believe that for the right task and in the right hands it can reduce the laboriousness dramatically.

Another reason is the desire to make SObjectizer a ready-to-use software product. For a long time SObjectizer was an internal tool with all expected consequences: absence of good examples and documentations, incomplete features, a lack of time to fix some drawbacks of initial design and a lack of resources to pay a technical debt.

In 2013-2016 some of us got the time and ability to work on SObjectizer without significant limitations. And we used that ability to change SObjectizer, to make it a good-quality OpenSource product. We had a chance and we used that chance. Now anybody can look at the SObjectizer-5 and judge the results.

The current state and the future of the project

The project is live and is slowly evolving. Please note that SObjectizer has already incorporated a lot of features and now it contains much more than we initially wanted. So the question "does SObjectizer-5 need yet more?" is an open question. With a probable answer "No".

For the last two or three years we add a new feature into SObjectizer (or so5extra) only if there is a need for that. When somebody asks us. Or we encounter a case in our own work where we lack a SObjectizer's feature.

And we are open.

If you want to see something in SObjectizer let us know. If a new feature is really needed we will try to add it to SObjectizer.

It's hard to speak about the future. But now it seems that we'll make SObjectizer-5 better by small steps. With great respect to backward compatibility. It will be good to keep next versions of SObjectizer-5.7 backward compatible with v.5.7.0 for four or five years from now (and maybe more).

And we don't see a need to start a new branch (like 5.8) for the near feature. So our plans for the future are related to the evolution of the 5.7 branch only. It also means that we haven't intention to switch from C++17 to new C++ standards during the lifetime of SObjectizer-5.7.

Acknowledgements

We want to say thanks for various people who helped us with the development of SObjectizer: some of you helped with porting SObjectizer to new platforms, some of you helped by reporting bugs and providing patches, some of you told how SObjectizer's internals can be improved, some translated SObjectizer's docs and made corrections for our written materials, some allowed us to give talks on conferences like CoreHard C++ and C++ Russia. Many thanks to all of you!

We also want to say thanks to all who selected SObjectizer for solving real-life problems. You took a risk of choosing an almost unknown tool from unknown developers from a small town in ex-USSR. You could select something different, something more popular and PRed. But you choose us. And that encourages us to go forward. Thank you!