Menu
▾
▴
bigdata-developers
|
From: Bryan T. <br...@sy...> - 2010-09-08 12:19:36
|
Brian,
I am wondering how we should construct and expose for configuration ServerSockets for custom data transfer services. Right now we have two such services which open ServerSockets to do NIO operations on ByteBuffers. These are the HA write replication pipeline and the ResourceService used to send shards around. I am currently extending the ResourceService to also ship around the intermediate results during distributed query evaluation.
Reading the javadoc, it would appear that we should be using the following form of the ServerSocket contructor:
public ServerSocket(int port, int backlog, InetAddress bindAddr) throws IOException;
This allows us to specify the port (or pass 0 for a random port) and to specify the local IP address on which the service will communicate. The backlog can be specified as 0 to use the default. For both of these services the port may be random since it will be disclosed by an RMI message, but it could also be nice to have the service talking on a configured port (for example, in order to simplify firewall configuration).
It seems that the pattern in use right now to specify the INetAddress is:
InetAddress.getByName(
NicUtil.getIpAddress("default.nic"/*systemPropertyName*/, "default"/*defaultNicName*/, false/*loopback*/)
);
Presumably, the "default.nic" value could be replaced by the name of a different system property if we wanted to bind high volume data transfers onto a specific. It that is the intention, then perhaps we should define an alternative system property name for those cases and begin to use it at the appropriate locations in the code base.
However, this still leaves unspecified a pattern to configure the port for the service. Would you envision a "PortUtil" (or an extension to NicUtil) which uses a similar pattern specifying the system property name and the default port name (or port number)?
Thanks,
Bryan
|
|
From: Bryan T. <br...@sy...> - 2010-09-08 12:31:52
|
I am also curious if there is a difference between an "ephemeral port" and a "free port". Ephemeral ports appear to be ports in a specific range (which might vary by the OS) and which are used for short lived service connections. If the service is to be long lived, should the free port be drawn from a different port range?
Thanks,
Bryan
> -----Original Message-----
> From: Bryan Thompson [mailto:br...@sy...]
> Sent: Wednesday, September 08, 2010 8:19 AM
> To: big...@li...
> Subject: [Bigdata-developers] How to configure the port and
> IP address for ServerSockets using NicUtil?
>
> Brian,
>
> I am wondering how we should construct and expose for
> configuration ServerSockets for custom data transfer
> services. Right now we have two such services which open
> ServerSockets to do NIO operations on ByteBuffers. These
> are the HA write replication pipeline and the ResourceService
> used to send shards around. I am currently extending the
> ResourceService to also ship around the intermediate results
> during distributed query evaluation.
>
> Reading the javadoc, it would appear that we should be using
> the following form of the ServerSocket contructor:
>
> public ServerSocket(int port, int backlog, InetAddress
> bindAddr) throws IOException;
>
> This allows us to specify the port (or pass 0 for a random
> port) and to specify the local IP address on which the
> service will communicate. The backlog can be specified as 0
> to use the default. For both of these services the port may
> be random since it will be disclosed by an RMI message, but
> it could also be nice to have the service talking on a
> configured port (for example, in order to simplify firewall
> configuration).
>
> It seems that the pattern in use right now to specify the
> INetAddress is:
>
> InetAddress.getByName(
> NicUtil.getIpAddress("default.nic"/*systemPropertyName*/,
> "default"/*defaultNicName*/, false/*loopback*/)
> );
>
> Presumably, the "default.nic" value could be replaced by the
> name of a different system property if we wanted to bind high
> volume data transfers onto a specific. It that is the
> intention, then perhaps we should define an alternative
> system property name for those cases and begin to use it at
> the appropriate locations in the code base.
>
> However, this still leaves unspecified a pattern to configure
> the port for the service. Would you envision a "PortUtil"
> (or an extension to NicUtil) which uses a similar pattern
> specifying the system property name and the default port name
> (or port number)?
>
> Thanks,
> Bryan
>
> --------------------------------------------------------------
> ----------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
|
|
From: Brian M. <btm...@gm...> - 2010-09-09 13:41:43
|
On Wed, Sep 8, 2010 at 8:18 AM, Bryan Thompson <br...@sy...> wrote:
I am wondering how we should construct and expose for configuration
> ServerSockets for custom data transfer services. Right now we have two such
> services which open ServerSockets to do NIO operations on ByteBuffers.
>
> public ServerSocket(int port, int backlog, InetAddress bindAddr) throws
> IOException;
>
> This allows us to specify the port (or pass 0 for a random port) and to
> specify the local IP address on which the service will communicate. T
> it could also be nice to have the service talking on a configured port (for
> example, in order to simplify firewall configuration).
>
It seems that the pattern in use right now to specify the INetAddress is:
>
> InetAddress.getByName(
> NicUtil.getIpAddress("default.nic"/*systemPropertyName*/,
> "default"/*defaultNicName*/, false/*loopback*/)
> );
>
> Presumably, the "default.nic" value could be replaced by the name of a
> different system property if we wanted to bind high volume data transfers
> onto a specific.
> It that is the intention, then perhaps we should define an alternative
> system property name for those cases and begin to use it at the appropriate
> locations in the code base.
>
Yes, "default.nic" could be replaced with a different value.
That value was created as a special value when you requested
that a mechanism be added for telling the configuration that you
would prefer that the system's default nic be used, whatever it
may be, rather than a specific nic name. This was needed because
the default nic on Windows cannot generally be know a priori, and
may actually change upon reboot. Whereas the use of this
'default.nic' mechanism is very convenient for development, it is
not necessarily the best solution for real deployment scenarios;
where there is quite a bit more control of the environment, where
there may be multiple nics one wishes to configure, and where
a priori knowledge can be exploited to make life easier for the
deployer in the data center.
For example, if you look at my personal branch, you will see a class
named ConfigDeployUtil that is used in each smart proxy's config
file to retrieve things like the groups or nics or ports, from a single,
top-level deployer-oriented config file that is in the form of a
name=value text file rather than a jini config (which deployer's seem
to prefer over the jini config format). The ConfigDeployUtil class
makes certain assumptions about its environment. So, although that
class is not very general purpose, it's intent is to shield the deployer
from the jini config files; thus making the life of the deployer much
simpler.
I wouldn't necessarily expect ConfigDeployUtil to be used with the
ServicesManagerService mechanism. And there are numerous
options for what may be used to replace the "default.nic" parameter
above; setting system properties being one such option.
> However, this still leaves unspecified a pattern to configure the port for
> the service. Would you envision a "PortUtil" (or an extension to NicUtil)
> which uses a similar pattern specifying the system property name and the
> default port name (or port number)?
>
I don't think a utility such as a "PortUtil" is necessary. The purpose of
the NicUtil was to provide code that can be executed from either
the config file itself, or from other code in the system; allowing one
to retrieve IP or Inet addresses based on a given network interface
name. This provides a way then, for avoiding the configuration of
specific IP addresses; instead configuring nic names, which vary far
less greatly than IP addresses do.
With respect to port numbers, it seems like a simple 'port = value'
should suffice; whether you're using jini configs or any other sort
of config file.
BrianM
|
|
From: Sean G. <sea...@no...> - 2010-09-09 15:57:20
|
On the PortUtil topic:
It is customary for applications to use well known ports for well known
protocols (http, ssh, etc), but use sequentially offset ports for each
custom protocol. Typically the default starting port is one a regular
user could bind on most platforms, somewhere between 1025 and 9999. If
the protocol only supports a single connection between two machines it
is also customary for the initiator to bind both local and remote ports
to the same number. This is common with peer-to-peer and routing
protocols, but client-server protocols tend to use 'random' ports for
the local port.
Something like:
int BIGDATA_STARTING_PORT =
Integer.parseValue(System.getProperty("bigdata.starting.port","5530"));
int BULK_TRANSFER_PORT = BIGDATA_STARTING_PORT + 0;
int HA_HEARTBEAT_PORT = BIGDATA_STARTING_PORT + 1;
int HA_DOWNSTREAM_PORT = BIGDATA_STARTING_PORT + 2;
These conventions allow network administrators to quickly setup firewall
rules using simple constructs like 'allow all traffic between
application nodes with a destination port between 5530 and 5532'.
One tricky bit is that this makes running multiple instances of the same
app on a single box difficult. Getting each app to bind to a different
port range is a simple config problem, but knowing the ports for other
apps is more difficult. Often this solved by creating globally shared
config that all apps can read. I personally find this approach
distasteful since globally shared state usually impacts scalability and
fault tolerance. Since BigData uses JINI, a more robust approach is
for each listener to publish port info as a JINI attribute in the
registry. One step better is for listeners to publish smart proxies,
which can completely insulate clients from the listener's ip, listening
port, wire protocol, etc...
Sean
On 09/09/2010 09:41 AM, ext Brian Murphy wrote:
> On Wed, Sep 8, 2010 at 8:18 AM, Bryan Thompson <br...@sy...
> <mailto:br...@sy...>> wrote:
>
> I am wondering how we should construct and expose for
> configuration ServerSockets for custom data transfer services.
> Right now we have two such services which open ServerSockets to
> do NIO operations on ByteBuffers.
>
> public ServerSocket(int port, int backlog, InetAddress
> bindAddr) throws IOException;
>
> This allows us to specify the port (or pass 0 for a random port)
> and to specify the local IP address on which the service will
> communicate. T
>
> it could also be nice to have the service talking on a configured
> port (for example, in order to simplify firewall configuration).
>
>
> It seems that the pattern in use right now to specify the
> INetAddress is:
>
> InetAddress.getByName(
> NicUtil.getIpAddress("default.nic"/*systemPropertyName*/,
> "default"/*defaultNicName*/, false/*loopback*/)
> );
>
> Presumably, the "default.nic" value could be replaced by the name
> of a different system property if we wanted to bind high volume
> data transfers onto a specific.
>
>
> It that is the intention, then perhaps we should define an
> alternative system property name for those cases and begin to use
> it at the appropriate locations in the code base.
>
>
> Yes, "default.nic" could be replaced with a different value.
> That value was created as a special value when you requested
> that a mechanism be added for telling the configuration that you
> would prefer that the system's default nic be used, whatever it
> may be, rather than a specific nic name. This was needed because
> the default nic on Windows cannot generally be know a priori, and
> may actually change upon reboot. Whereas the use of this
> 'default.nic' mechanism is very convenient for development, it is
> not necessarily the best solution for real deployment scenarios;
> where there is quite a bit more control of the environment, where
> there may be multiple nics one wishes to configure, and where
> a priori knowledge can be exploited to make life easier for the
> deployer in the data center.
>
> For example, if you look at my personal branch, you will see a class
> named ConfigDeployUtil that is used in each smart proxy's config
> file to retrieve things like the groups or nics or ports, from a single,
> top-level deployer-oriented config file that is in the form of a
> name=value text file rather than a jini config (which deployer's seem
> to prefer over the jini config format). The ConfigDeployUtil class
> makes certain assumptions about its environment. So, although that
> class is not very general purpose, it's intent is to shield the deployer
> from the jini config files; thus making the life of the deployer much
> simpler.
>
> I wouldn't necessarily expect ConfigDeployUtil to be used with the
> ServicesManagerService mechanism. And there are numerous
> options for what may be used to replace the "default.nic" parameter
> above; setting system properties being one such option.
>
> However, this still leaves unspecified a pattern to configure the
> port for the service. Would you envision a "PortUtil" (or an
> extension to NicUtil) which uses a similar pattern specifying the
> system property name and the default port name (or port number)?
>
>
> I don't think a utility such as a "PortUtil" is necessary. The purpose of
> the NicUtil was to provide code that can be executed from either
> the config file itself, or from other code in the system; allowing one
> to retrieve IP or Inet addresses based on a given network interface
> name. This provides a way then, for avoiding the configuration of
> specific IP addresses; instead configuring nic names, which vary far
> less greatly than IP addresses do.
>
> With respect to port numbers, it seems like a simple 'port = value'
> should suffice; whether you're using jini configs or any other sort
> of config file.
>
> BrianM
>
|
Thanks for helping keep SourceForge clean.
X