Menu

Web Service engine

2016-03-28
2016-04-27
  • Barry Evans

    Barry Evans - 2016-03-28

    Hi all,
    As part of my day job I've been running a team playing with some Micro Services stuff. As part of the work I wanted a side project to experiment with some things which I got carried away with.

    I have taken the core JAR from the Mars-Sim project and use Spring Boot and other projects to create a web service API onto the simulation engine. The simulation runs in the background and the updates can be seen via some endpoints as JSON responses. This simple service has Settlements and Persons available plus a basic information on the Simulation. It can also be paused and resumed.

    The code is in BitBucket as a complet Maven project that can be run.
    It contains an embedded Swagger UI HTML interface that provides an easy to use interface.

    https://bitbucket.org/bevans2000/mars-sim-service

    I've also deployed a working version onto AWS EC2 to see how long the simulation runs. It is running on a t2.micro so only has very limited resources.
    http://ec2-52-49-180-86.eu-west-1.compute.amazonaws.com:8080/swagger-ui.html

    Tell me what you think.

    Barry

     
  • manny kung

    manny kung - 2016-03-29

    Hi Barry,

    It's interesting work that you just make it possible for mars-sim to run as a web service !

    Do you have more examples/experiments that we can set up running ?

    What do you think we can play with next?

    -Manny

     
  • Barry Evans

    Barry Evans - 2016-03-29

    Yes quite a bit can easily be achieved without much coding.

    I think completing the end points for Person & Settlement would be good step. So something like /settlements/{$id}/buildings or /persons/{$id}/health. These entities have quite deep object graphs.

    The ideal next step would be for someone to create a web ui using a modern JavaScript frameworks like Angular or Ember. This would allow a songle simulation instance to be accessed by multiple people across the internet.

    Based on that people could have accounts and be put in control of certain Units. The key is the simulation engine is very powerful and accurate, way better than some of the simulation apps you find on iTunes. So maybe it could be turned into an online presence that people can login in and take part.

    One problem i see is how everything is stored. A better way to store the state is needed than using object serialisation. But if there is a RESTful API available the backend elements can be changed over time.

    One mad idea I had was to create a Twitter account and get the engine to generate tweets when something interesting happen. Spring has a Twitter integration project and I have used this before to autogenerate posts.

    Barry

     
  • manny kung

    manny kung - 2016-03-29

    his would allow a songle simulation instance to be accessed by multiple people across the internet... Based on that people could have accounts and be put in control of certain Units.

    I can imagine that this web version of mars-sim opens up the possibility of (1) co-op feature and (2) persistent world.

    So I have no doubt to believe that this new developmental path will take mars-sim to a new height !

    Last year, I've wanted to give mars-sim multiplayer option but my approach so far limits it to running within a home network and is not robust.

    If we ever set up a new story mode and code an role/rank assignment system that let a user play as a settler that has a rank/role out of a command structure or a hierachy, each user could log in as the 1st person mode instead of being in the typical 3rd person "God"-mode in a lot of strategy games.

    I imagine each user will have unique advantage to perform different tasks. We can even make a user's tasks dependent upon the assistance received from other users or NPCs.

    So let's spell out more right here or in the developer/user mailing list the things what we need to do exactly to get there.

     
  • Barry Evans

    Barry Evans - 2016-03-29

    Let me have a go over the next few days to flesh out the person and settlement end points. But we would need some web ui to be developed that utilised the web service. This should ge something like React or Angular coupled with a responsive design like Bootstrap so it could be usable from mobiles as well as laptops. Whilst i have played with Angular i am just a novice.

    Or a mobile native app.

    I still think there could be a problem with the persistency aspect of the simulation state. Serialization is fine for single user but muliuser needs something more robust. Plus thinking ahead it should be possible to run the simulator engine in a seperate VM from the web service, allowing the latter to scale with multiple instances.

     
  • manny kung

    manny kung - 2016-03-29

    Serialization is fine for single user but muliuser needs something more robust.

    Yes serialization will have to be reworked.

    For one, we'll need to lower the network bandwidth. In the last build r3608, I just switched from using GZIP to LZMA2 to compress the save file. Under gzip, a new sim costs ~620kB. With LZMA2, it's down to ~370kB now. However, it's still unrealistic to sync, 370kB world data between, say 10 users, once a few frames.

    We've been using Java's ObjectInputStream and ObjectOutputStream which is binary. If we swtich to JAXB, will it solve the problem ?

     
  • Barry Evans

    Barry Evans - 2016-03-30

    If the application uses a web based interface then there is no need to sync the large save file between the users. Users would just be downloding JSON responses into their browser.

     
  • Lars Christensen

    I think the aspects of moving MSP from client-side Java to a web app are very very interesting. I see CS Java going the same way as Flash, Sliverlight etc. and I'd be happy to not require people to use any other tool than their HTML5-capable browser.

     
  • manny kung

    manny kung - 2016-03-31

    if we're going with server side java, how do we pay for the traffic/usage ? there's no telling how much traffic it will be and is there a free open source hosting option rather than AWS ?

     
  • manny kung

    manny kung - 2016-04-01

    In terms of the potential of utilizing HTML/CSS UI,

    we could make use of a lot of those nice UI templates out there with free animations such as this CSS weather animation for mars-sim android app:
    http://codepen.io/davidkpiano/full/ByNPQw/

    I also found this nice short and sweet solar system animation written in pure css. Can be adapted to mars-sim for web:
    http://codepen.io/kowlor/pen/ZYYQoy

    A lot more on dribbble:
    https://dribbble.com/search?q=ux+animation

    I'm nearly giving up building new UI using JavaFX--in which we have to build UI from the ground up-- and go vote for HTML/javascript/CSS--where a lot of good-looking templates are readily accessible that provide better UI/UX.

     
  • Barry Evans

    Barry Evans - 2016-04-01

    The first important aspect is how to achieve the change to a client server approach. Creating the client running inthe browser is a significant undertaking. What technology to use? Ruby on Rails, JavaScript framework?

    The current engine, whilst massively impressive in its detail and modelling, needs to be able to run for days without problem.

    Hosting the back end service is a simpler activity. Either a IaaS like AWS is easy but something like Heroku or CloudFoundry is a great match for Java Spring Boot. The cost of running these is not excessive to get something running. If it takes off with many users then thats a great problem to have; maybe Google Ads or getting people to donate.

     
  • manny kung

    manny kung - 2016-04-01

    Hi Barry,

    Will you consider using a framework to bridge between JVM and Web UI ?

    What do you think about these two frameworks, namely, Vaadin and Dukescript ?

    Using either one of them allows us to keep our Java engine for the backend while giving us a new lease of life in our UI design.

    Comparison between them
    https://www.javacodegeeks.com/2015/08/hello-world-in-vaadin-dukescript.html

    To me, I want to vote for Dukescript since I can readily use javascript to build UI and thus having easy access to all the goodies out there.


    1. Vaadin : has a lot of add-on's
      https://vaadin.com/features

    See https://vaadin.com/license :
    Vaadin Framework is distributed for free as Open Source, and it is licensed under permissive Apache License, version 2.0. This means for example that you may freely download and use Vaadin Framework for commercial, personal and non-commercial purposes. You may also modify the framework, use it as part of your own software and distribute it.

    Comparison between Vaadin with others:
    https://vaadin.com/comparison

    From https://vaadin.com/faq
    "The browser side rendering engine of Vaadin is implemented using GWT. At runtime there are no GWT dependencies in the server side, and the browser side is pure Javascript with no dependencies to Google. The javascript code shipping with Vaadin is precompiled, and no GWT compilation is required when using them. "


    1. Dukescript
      https://dukescript.com/

    http://www.i-programmer.info/news/80-java/7828-dukescript-java-everywhere-again.html

    https://www.javacodegeeks.com/2015/08/java-everywhere-write-once-run-anywhere-with-dukescript.html
    "Currently, DukeScript supports the various desktop platforms, as well as iOS, Android, and the browser. The available Maven archetypes create, for each supported platform, a separate subproject."

     
  • Barry Evans

    Barry Evans - 2016-04-01

    I'm no web UI expert, my day job is about creating large enterprise solutions using cooperating services or micro services as they are called these days. I think its important to have a stable and well documented service that any client could use. So i feel if we do want a client server answer the client side should use a technology that allows rapid development in that sphere and independent of the back end.

    However my personal opinion is a JavaScript framework like Ember or React is the best choice; reuse what is already out there. These are widely used and certianly on the rise it terms of popularity. Previously I have played with AngularJS which I really like but the new version appears very different and has a lot of doubters out there. Apache Condova also looks impressive. But i dont have the knowledge to create such a web UI from scratch currently.

    I think what i am saying it would be good to know someone who is an expert in this field and get them to write it :-)

    I intent to develop the back end service further over the next few weeks and see how that works out delivering the complex entities like Settlements over JSON?

     
    • manny kung

      manny kung - 2016-04-02

      a JavaScript framework like Ember or React is the best choice; reuse what is already out there. These are widely used and certianly on the rise it terms of popularity.

      Sure. it's better we take advantage of a javascript framework.

      Can we refine our ideas further and lay out a list of objectives as we are going to fork our development to create a mars-sim version for web ?

      Also, will it simply be carrying the multiplayer feature (in addition to the single player/standalone) OR becoming a Massively Multiplayer Online (MMO) type of game/simulation ?

       
  • Barry Evans

    Barry Evans - 2016-04-04

    I have pushed a later version of the web service code to my BitBucket repo over the weekend. it covers some more of the Settlement endpoint which is probably the largest as it has the largest object graph.
    I plan to add the container concept so it can show what is stored in a Unit.

     
    • manny kung

      manny kung - 2016-04-05

      I can see you created 3 folders, namely, \controllers, \mappers and \model in \src\main\java\org\mars_sim\restws and with a bunch of new java classes in them.

      I'm reading tutorials such as http://rest.elkstein.org/2008/02/using-rest-in-java.html

      It'll take a while for me to comprehend how everything works and how you implement REST in mars-sim.

      btw, I've seen how people build a backend server app in node.js (or if just a very simple web page they use sail.js). Then they use angular to build the front end Web UI.

      So, in our case, how easy/difficult is it for us to build a server app (aka mars-sim server) that can be run in any PC?

      This way, any users can deploy their own mars-sim on their own network cuz we should probably first make mars-sim multiplayer capable.

      As it stands right now, even after we've converted everything in mars-sim-core to a web service, It still lack multiplayer-specific features.

      Besides having to buid a Web UI from the ground up, we need to define different levels of interactions. Say, if multiple users log in, how do we assign what they can and cannot do to make the co-op experience to execute in a smooth way?

      So, I'm thinking we should first lay out a list of multiplayer features we want to see and implement in the next few releases.

      After we test out these new multiplayer features for a year or two, mars-sim will be matured enough to move to the next level of a becoming MMO.

      By then we would have laid out a full plan for marketing mars-sim and get support to be funded and host it in AWS.

       
  • Barry Evans

    Barry Evans - 2016-04-05

    I see the eventual solution should have 3 repos:
    1. existing mars sim code base for the core simulator and maybe a desktop version
    2. the REST WS code base that pligs on to the core
    3. a seperate web client using just the published interface
    Keeping the client seperate allows multiple clients to be developed

    The readme in the BitBucket repo explains how to use the maven spring boot plugin to run the server locally.

    How do you want to capture the potential multi user features? A page on the Wiki in Source Forge. Or use something like Trello to create a board to manage some cards.

    I have now added some items and resources end points on Persons and Settlements.

     
    • manny kung

      manny kung - 2016-04-06

      I'll fresh out some multiplayer features I have in mind in a wiki page.

      Yes I've also just set up a team called mars-sim in trello. see https://trello.com/marssim. Are you still in mars-sim developer's mailing list? Let me know your name in trello and/or email address so that I can add you as a member and we can add more cards.

      Meanwhile, I'll add that maven spring boot plugin into mars-sim's pom.xml.

      Based on http://docs.spring.io/spring-boot/docs/1.3.3.RELEASE/maven-plugin/usage.html , it will automatically repackages the jar that is built during the package phase of the Maven lifecycle for spring boot.

      Do you have the latest development environment for 3.8-snapshot set up up running in eclipse ?

      We're coming very close to wrapping up everything for the final release. May be we can start working off the 3.8-snapshot source code, rather than 3.7.

      We could create two additional maven modules such as called mars-sim-server and mars-sim-webui. If we can keep everything under one single repo, it'll be nice. Having one repo rather than multiple repos simplifies how to sync up the new changes.

      Can you DEMONSTRATE THE EXACT STEPS it would take for me to integerate spring tool into mars-sim so that I may load it up as a server app on my PC (not in AWS) ?

      I've read your readme at https://bitbucket.org/bevans2000/mars-sim-service a few times but I don't know how to connect all the pieces together yet. Any links and/or examples would appreciate !

      If we are fast, we could quickly build a web ui to run in a broswer with just a few elements/things to look up. It will be like creating an "observer mode" to just observe how things are going in a running settlement.

       
  • Barry Evans

    Barry Evans - 2016-04-06

    My account on Trello is @barryevans4. I forgot that SourceForge uses SVN; haven't used it for years. I only use Git these days. Can you my sourceforge account back to the access list.

    I will add some ore notes to the readme at weekend when I have some time. But in short:
    1. clone the Git repo from BitBucket
    2. run mvn spring-boot:run
    3. this will build everything needed and start the server on port 8080
    4. go to http://localhost:8080/swagger-ui.html and you get that to access the WS endpoints or go directly.

    So you are thinking to add new top level folders for a service and web-UI

     
  • manny kung

    manny kung - 2016-04-26

    Hi Barry,

    Can you put together quickly a working app (say with just a couple of java class) and run on spring and deploy it locally and run in with just a few basic web ui controls? This way, I can observe how different pieces works together.

    I've sent you some mails on your SF account under your name. Do you by any chance receive them?

    What's your github username? I'd like to add you onto our site.

    Manny

     
    • Barry Evans

      Barry Evans - 2016-04-27

      Yes I got the emails and sent you a reply. I'm not a lover of SourceForge and would much prefer to use GitHub.

      In the dev mailing list yesterday I said I would try and get something out together at weekend.

      Barry

      Sent from my iPad

      On 26 Apr 2016, at 23:46, "manny kung" mkung@users.sf.net wrote:

      Hi Barry,

      Can you put together quickly a working app (say with just a couple of java class) and run on spring and deploy it locally and run in with just a few basic web ui controls? This way, I can observe how different pieces works together.

      I've sent you some mails on your SF account under your name. Do you by any chance receive them?

      What's your github username? I'd like to add you onto our site.

      Manny

      Web Service engine

      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/mars-sim/discussion/open_discussion/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       
  • manny kung

    manny kung - 2016-04-27

    Can you resend ? sorry I don't see your reply. it must have been deleted accidentally.

    We have a github site at https://github.com/mars-sim for backup but SF is the main site.

    I can add you to our core team if you give me your github name in email.

    We're sticking to svn for now since git is not that perfect and arguably svn is more dependable and work well for us.

    SF is not bad at all. It just needs modernization of its UI and be more transparent.

    On the other hand, github has over 1M+ issues to fix:
    https://news.ycombinator.com/item?id=11578191

     

    Last edit: manny kung 2016-04-27

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.