I am looking for documentation/samples on how to use JavaFBP APIs. I came across your project when I was reading about Java based DataFlow programming languages.
I looked at pervasive's datarush API and looking for alternatives to them.
Does JavaFBP API make use of Multi-core CPUs and help us do parallel programming in Java? If so, please point me to the documentation.
Thanks in advance,
Yes, JavaFBP uses multiple Java Threads, so, depending on the JVM implementation, it should use multiple cores. The book is on http://www.jpaulmorrison.com/fbp (about half-way down) - and the documentation for JavaFBP is at http://www.jpaulmorrison.com/fbp/jsyntax.htm
Feel free to email me if you have questions.
paul dot morrison at rogers dot com
Thank you very much Paul. I will keep you posted on my progress.
I finally got a chance to use JavaFBP to do few experiments - to see if architecting the solution using FBP would help me process data faster. It is amazing how a 182KB jar file could so much. I have few quick questions.
How do we control the tracing feature ?
Was wondering how we could switch off tracing and if there are any tracing-levels defined somewhere.
I am using JavaFBP-2.3.jar and DrawFBP-2.3.jar.
Would greatly appreciate your help here.
JavaFBP is up to 2.4 - I'm not sure when the tracing was upgraded, but 2.4 generates a single trace file for the network, and a separate one for each subnet (I've forgotten if 2.3 does too). Feel free to suggest improvements - I'm not much of a tracing expert! To switch off tracing altogether, set the boolean variable called <b> tracing</b> to false in Network in package com.jpmorrsn.fbp.engine . I know how to set properties in C#, but I didn't know how to set them in Java! Switching off tracing certainly speeds things up considerably! You could also read the chapter on performance - http://www.jpaulmorrison.com/fbp/perform.htm - if you haven't done so already.
Hope this helps - let me know how it goes! Paul
Thanks Paul. I turned off the trace per your email. The program used to run for about 32 minutes earlier - with tracing. Now it took 21 minutes to do the same job.
BTW - I am comparing JavaFBP with just plain simple Java-Class implementation of the same logic. It has been a good exercise so far. I will publish all the results once I am done. Should be done by eod today. I will try to get some documentation ready too - that would probably help the visitors on your site. I will probably come back with more questions on JavaFBP too :).
Here is what I was trying to do. I have a text file with data in the following format. This text file has 560K lines.
1001, "1A-3A, 20B-25B"
1002, "2AA-4AA, 5BB-8BB"
I need to read the above file and create another file in the following format. As you can see the target file will have more lines. Data with error conditions need to be written into another file.
I implemented the above in 3 different ways (all three programs shared the same core logic - with few system.out.println statements of course).
1)With JavaFBP - Single Process way - refer to Single_Process_Example.png
2)With JavaFBP - Using Multi-plexing - refer to Multi_Process_Example.png
3)With Plain Java Class reading,processing and writing from the same Java Class
Here are my findings - (swtiched off the tracing). The following programs were run
on a Intel Xeon at 3.0 GHZ - 4 CPU Windows 2003 Server OS - with 6 GB RAM)
1)With JavaFBP - Single Thread Processing (21 minutes)
2)With JavaFBP - Using Multi-plexing (30 minutes)
3)With Plain Java Class reading,processing and writing from the same Java Class (17 minutes)
I was expecting a performance improvement when I did the multiplexing but it degraded as you can see. My goal was to use multiplexing so that we have different threads working on different lines coming in from load balancer - thereby decreasing the elapsed time to process the source file. The ultimate goal was to see shorter processing time than the Single plain java-class by using multi-threading.
Is multi-plexing not helping us here because there are some seconds lost in load_balancing upstream and consolidating the packets downstream? I was trying to see if JavaFBP can help me process data faster using multiplexing since we have more CPU power available from the new multi-core, mutli-CPU servers. Please let me know if there is a different way to approach the above problem.
That's very interesting! I can't see the png files, but I assume the multithreaded version uses a load balancer and multiple instances of your "splitter". Given the simplicity of the task, it is not too surprising that JavaFBP takes a bit longer than a single Java class. However, the multithreaded version should indeed be able to take advantage of the 4 processors - but you imply that you had to "consolidate" the packets downstream, which would end up costing quite a bit. I see in your problem description that you want to preserve the original sequence in your output, but I am wondering if you really need to do that. In my experience, we only need to maintain sequence if humans are going to read it - or you need a fixed sequence so you can compare the file with another one. In my book I advise against trying to recombine "split" files as it is usually not worth the overhead!
Log in to post a comment.
Sign up for the SourceForge newsletter:
You seem to have CSS turned off.
Please don't fill out this field.