I started making changes to implement back the stream table. One stream is used to hold stream table that maps streamId to first segment position. For now the table is a simple table which is also fully chached in memory but the idea is to later replace it's structure with B+ tree.
When I did the changes I had many problems. I had to fix bugs which where difficult to locate but still it crashed with my testing tool. So one day I just rewrote the core part from scratch. The whole engine that allocates segments is now new. I kept some of the parts like StorageStream and transaction system.
There where some bugs that took some time to discover but now it's finally done. I ran my test tool for longer periods and there where no crashes.