3D items in "*.MIS" (ObjVec chunk) have different IDs than in "OBJ.CRF" ?

JFRegnault
2014-04-07
2014-11-27
  • JFRegnault

    JFRegnault - 2014-04-07

    Hi, I'm currently trying to extract SystemShock2's 3D levels into a more recent format visible in a standard 3D viewer. No interactions, just static levels.
    Until now I managed to :
    - extract all 3D items templates from the file "OBJ.CRF" (.BIN entries)
    - extract 3D levels from "
    .MIS" files (using their chunks "WRRGB/TXLIST/FLOW_TEX").

    Now my problem is the chunk "ObjVec" of .MIS files : it gives the position of all items in a level, but without their real IDs ! I mean, it seems that the IDs from "ObjVec" must somehow be mapped to the real IDs used in "OBJ.CRF".

    Did I miss something, is there a trick ? Thank you for giving me any clue

    JF

     
    • Filip Volejnik

      Filip Volejnik - 2014-04-07

      Hello,

      You're right. The way dark works is as follows:

      • objects have unique, numerical ids.

      • negative ids belong to archetypes - object templates

      • positive ids belong to instances

      Each object is composed of properties. Some properties are directly
      assigned to objects, some are inherited, based on the way the property
      inherits (there are different inheritance rules for different properties).

      So, for your problem, you'll need to handle the object system to some
      degree. Mainly the ModelName property and its inheritance (this one always
      inherits). Not many objects will have the ModelName property directly set,
      so you'll have to handle object links as well, in this case metaproperty
      links which link object to their archetypes.

      You'll also probably need some other properties to be 100% correct -
      ModelScale, RenderAlpha, RenderType, HasRefs (ZBias...). You can find the
      property definitions in src/services/render.

      I'd be interested to find out where you end up with your project, be sure
      to keep me informed :)

      Volca
      Dne 7. 4. 2014 18:09 "JFRegnault" jfregnault@users.sf.net napsal(a):

      Hi, I'm currently trying to extract SystemShock2's 3D levels into a more
      recent format visible in a standard 3D viewer. No interactions, just static
      levels.
      Until now I managed to :
      - extract all 3D items templates from the file "OBJ.CRF" (
      .BIN entries) - extract 3D levels from ".MIS" files (using their chunks
      "WRRGB/TXLIST/FLOW_TEX").

      Now my problem is the chunk "ObjVec" of .MIS files : it gives the position
      of all items in a level, but without their real IDs ! I mean, it seems that
      the IDs from "ObjVec" must somehow be mapped to the real IDs used in
      "OBJ.CRF".

      Did I miss something, is there a trick ? Thank you for giving me any clue

      JF

      3D items in "*.MIS" (ObjVec chunk) have different IDs than in "OBJ.CRF" ?https://sourceforge.net/p/opde/discussion/517377/thread/39b04d8f/?limit=25#df7e

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/opde/discussion/517377/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       
  • JFRegnault

    JFRegnault - 2014-04-07

    Hi Filip,
    Thank you for answering so fast, now I'll try to figure out the subtilities of your precisions (archetype / property inheritance / object links). I guess I'll need much efforts to manage these damned 3D items ! I may have to read parts of OPDE source code but I'm not used to C++. I'll tell you if I finally reach an acceptable result.
    Thanks again

    JF

     
    • JFRegnault

      JFRegnault - 2014-05-03

      Hi Philip,

      After your last post, I dived into OPDE source code. I even tried to implement parts of it in C#, but there are so many classes which are dependant from each other that I quickly found myself writing lines and lines of code (~5000) ... which is clearly nonsense as my goal wasn't to re-implement OPDE...
      Moreover, I inevitably added my own bugs all along, trying to simulate multiple inheritances (impossible in C#) and so on.
      Now I’m a bit ashamed to admit that I'm still standing at the same point: in a *.MIS file we have the "ObjVec" chunk giving all item Ids, which can then be found in other chunks such as "P$Position", "P$Scale" (and "L$MetaProp", "LD$MetaProp" : I still not fully understood the meaning of these ones : src id, dest id, concrete id, link id…)
      I can't link the Ids from these chunks to the 3D meshes from "OBJ.CRF" file, which actually have no Id, just a name (ex: "trash.bin").

      I still still hope I can avoid implementing the hierarchy of nodes (and all the listener system) to reach my goal, and that I can directly - even if not clean - spot the needed data directly into the game files.

      Maybe the solution is more obvious for you? If so I'd highly appreciate another (and hopefully last) helping hand/explanation/sign... anything that could unblock me.

      Anyway, I wish that you'll find time and motivation to achieve you own goal with OPDE, I'm looking forward to seeing it completed one day!

      JF

       
      • Filip Volejnik

        Filip Volejnik - 2014-05-05

        Hey!

        No problem, glad to help :)

        I think you may do it quite simply, just load the metaprop links into a
        multimap indexed by object id, and build a traversal function that will
        either output all inherited id's in order, or given a functor will call it
        on all ids. Then traverse this with all object ids and always look for
        first id that has the property in question set (remember you need more than
        one property to make it look okay, but modelname is the crucial). You will
        also need to use P$HasRefs as without it, some objects that should not be
        visible will be.

        something in the lines of this:

        multimap<int, MetapropLink=""> metaprops;

        ... should sort for each key by priority

        then the traversal

        bool traverse(int id, std::function<bool (int)=""> func) {
        foreach (auto e: metaprops[id]) {
        if (func(e.dst))
        return true;
        if (traverse(e.dst, func))
        return true;
        }
        return false;
        }

        ... then the functions for the properties
        map<int, ModelName=""> modelNameProp;

        struct AssignModelName {
        int id = 0;

        bool operator()(int id) {
            if (modelNameProp.find(id) != modelNameProp.end()) {
                this->id = id;
                return true;
            }
            return false;
        }
        

        }

        AssignModelName amn(objectId);
        if (traverse(objectId, assignModelName)) {
        ... we have the model name effective object id in the amn.id...
        }

        hope this helps :)

        2014-05-03 14:51 GMT+02:00 JFRegnault jfregnault@users.sf.net:

        Hi Philip,

        After your last post, I dived into OPDE source code. I even tried to
        implement parts of it in C#, but there are so many classes which are
        dependant from each other that I quickly found myself writing lines and
        lines of code (~5000) ... which is clearly nonsense as my goal wasn't to
        re-implement OPDE...
        Moreover, I inevitably added my own bugs all along, trying to simulate
        multiple inheritances (impossible in C#) and so on.
        Now I’m a bit ashamed to admit that I'm still standing at the same point:
        in a *.MIS file we have the "ObjVec" chunk giving all item Ids, which can
        then be found in other chunks such as "P$Position", "P$Scale" (and
        "L$MetaProp", "LD$MetaProp" : I still not fully understood the meaning of
        these ones : src id, dest id, concrete id, link id…)
        I can't link the Ids from these chunks to the 3D meshes from "OBJ.CRF"
        file, which actually have no Id, just a name (ex: "trash.bin").

        I still still hope I can avoid implementing the hierarchy of nodes (and
        all the listener system) to reach my goal, and that I can directly - even
        if not clean - spot the needed data directly into the game files.

        Maybe the solution is more obvious for you? If so I'd highly appreciate
        another (and hopefully last) helping hand/explanation/sign... anything that
        could unblock me.

        Anyway, I wish that you'll find time and motivation to achieve you own
        goal with OPDE, I'm looking forward to seeing it completed one day!

        JF

        3D items in "*.MIS" (ObjVec chunk) have different IDs than in "OBJ.CRF" ?https://sourceforge.net/p/opde/discussion/517377/thread/39b04d8f/?limit=25#12b0/78ec

        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/opde/discussion/517377/

        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/

        --
        Ja Filip

         
  • Filip Volejnik

    Filip Volejnik - 2014-05-05

    Oh the formatting...

    I seem to have made a trivial mistake:

    bool traverse(int id, std::function<bool(int)> func) {
        if (func(id))
            return true;
        foreach (auto e: metaprops[id]) {
            if (traverse(e.dst, func))
                return true;
        }
        return false;
    }
    
     
  • JFRegnault

    JFRegnault - 2014-10-31

    Hello Filip,
    After a long period of inactivity I'm trying to apply again the precise instructions that you gave me a few months ago.
    At first read, I still have the same doubt:
    - on one side we have all objects known by their names, from file OBJ.CRF => "TRASH.BIN", "TRAMCAR.BIN", ... "*.BIN" (approx. 1500 item names)
    - on the other side, I understand that the ModelName+Relations would give us an "effective id"
    => but I still can't connect these 2 informatons [int ID] versus [string itemName]. The ID doesn't seem to be the index in OBJ.CRF's collection of items. Aaargh!

    Therefore my scenes remain empty without any items inside, it's much frustrating.
    I opened almost every binary data file (looking for clues), and read as much OPDE C++ source code that I could possibly understand.
    But without success as far as now.

    Is there a way to map an "effective itemID" to a ".BIN filename" ?
    Or maybe my approach is still poor ?

    Regards,
    JF

     
    • Filip Volejnik

      Filip Volejnik - 2014-11-05

      Hey,

      sorry for the delayed reply, I am sort of lacking time these days.

      I understand your frustration, dark engine is a complicated beast. The
      callback nature of opde does not help either, because there is no direct
      way to observe all the side effects of loading operations.

      I think you understood one piece wrong, still. There is no index into the
      obj.crf (which is an unordered zip file anyway), the models are referenced
      by name in ModelName property.

      It is a rather complicated thing, you have to implement the link to get the
      object's model name. It works like this:

      For each object, you have it's id. Then you travel down the object
      hierarchy using the metaprop links to find all the object instances
      (including the abstract object templates with negative id's) and for all
      those, you search for the desired property with that ID (in this case, the
      property is modelname). It is essential you do this correctly, but for most
      cases it should be enough to travel bredth first (well relation also has a
      priority, but let's ignore that for now).

      Example: You have a chair with id 103. Metaprop link links it to abstract
      object -15. Abstract object -15 links to -32. Object -32 has ModelName
      property set. You read the string in that property to get the object's name.

      Note: You select the first object while traveling back in the hierarchy
      that possesses the property you seek.

      Is there a repository with your code? I might tell you what to change etc
      if there were.

      Hope this helps :)

      Filip

      2014-10-31 16:47 GMT+01:00 JFRegnault jfregnault@users.sf.net:

      Hello Filip,
      After a long period of inactivity I'm trying to apply again the precise
      instructions that you gave me a few months ago.
      At first read, I still have the same doubt:
      - on one side we have all objects known by their names, from file OBJ.CRF
      => "TRASH.BIN", "TRAMCAR.BIN", ... "*.BIN" (approx. 1500 item names)
      - on the other side, I understand that the ModelName+Relations would give
      us an "effective id"
      => but I still can't connect these 2 informatons [int ID] versus [string
      itemName]
      . The ID doesn't seem to be the index in OBJ.CRF's collection of
      items. Aaargh!

      Therefore my scenes remain empty without any items inside, it's much
      frustrating.
      I opened almost every binary data file (looking for clues), and read as
      much OPDE C++ source code that I could possibly understand.
      But without success as far as now.

      Is there a way to map an "effective itemID" to a ".BIN filename" ?
      Or maybe my approach is still poor ?

      Regards,
      JF


      3D items in "*.MIS" (ObjVec chunk) have different IDs than in "OBJ.CRF" ?
      https://sourceforge.net/p/opde/discussion/517377/thread/39b04d8f/?limit=25#0ecd


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/opde/discussion/517377/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

      --
      Ja Filip

       
      Attachments
  • JFRegnault

    JFRegnault - 2014-11-27

    Hi Philip,

    Thanks to your last post, I finally managed to add items in my 3D scenes (actually a few items are misplaced, I don't know why, but most of them are placed correctly). The result is acceptable for me.
    Also one of my problems is that I missed to decode SHOCK2.GAM file, which is just another "tag" file (like .MIS) and contains crucial informations...

    Characters are still missing (they don't seem to be technicaly stored like items), maybe later I'll try to figure out their locations.

    Thanks again for helping me, I uploaded the resulting .X files here: http://jfregnault.free.fr/SystemShock2/SS2.htm

    I hope you manage to complete OPDE one day!

    JF

     
    • Filip Volejnik

      Filip Volejnik - 2014-11-27

      Hey!

      Glad you made it after all. Congratulations. If you have any further
      questions, don't hesitate to ask :)

      Volca

      2014-11-27 2:53 GMT+01:00 JFRegnault jfregnault@users.sf.net:

      Hi Philip,

      Thanks to your last post, I finally managed to add items in my 3D scenes
      (actually a few items are misplaced, I don't know why, but most of them are
      placed correctly). The result is acceptable for me.
      Also one of my problems is that I missed to decode SHOCK2.GAM file, which
      is just another "tag" file (like .MIS) and contains crucial informations...

      Characters are still missing (they don't seem to be technicaly stored like
      items), maybe later I'll try to figure out their locations.

      Thanks again for helping me, I uploaded the resulting .X files here:
      http://jfregnault.free.fr/SystemShock2/SS2.htm

      I hope you manage to complete OPDE one day!

      JF

      3D items in "*.MIS" (ObjVec chunk) have different IDs than in "OBJ.CRF" ?
      https://sourceforge.net/p/opde/discussion/517377/thread/39b04d8f/?limit=25#f9bb


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/opde/discussion/517377/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

      --
      Ja Filip

       
      Attachments

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks