Re: [jgroups-users] Eclipselink cache coordination with JGroups in Docker in AWS

OK, I found out how to do this.

I'm going to write up a more detailed section in the manual and I'll 
also adjust the docker image of JGroups (below).

The following only applies to network=bridge; network=host doesn't 
require the following steps.

OK, so I started 2 (Amazon ECS-Optimized Amazon Linux AMI) instances A 
and B and then ran:

docker -it --rm --network=bridge belaban/jgroups

The IP addresses were:
                    A:                                B:

private eth0: 172.31.47.164   172.31.43.171
docker eth0: 172.17.0.2         172.17.0.2 // same virtual eth0 addresses

I also disabled all firewall rules on A and B using 'sudo iptables -F'. 
I don't think this was needed but anyway. Ditto for selinux if enabled.

With bridged networking, JGroups has to bind to the private docker eth0 
address, so it binds to 172.17.0.2:7800. Note that JGroups cannot bind 
to the private IP address of the EC2 instance, as this is NOT a valid 
interface!

However, the external address needs to be used to register the name, 
e.g. for A:
    <TCP
          external_addr="${EXTERNAL_ADDR:172.31.47.164}"
	 bind_addr="match-interface:eth0"
          bind_port="7800"  />
     <TCPPING initial_hosts="172.31.47.164[7800],172.31.43.171[7800]"/>
...
     <FD_SOCK external_addr="${EXTERNAL_ADDR:172.31.47.164}"
              start_port="9000"/>

In TCP, bind_addr will match eth0, which is the private docker eth0 
address of 172.17.0.2. The external_addr attribute points to the private 
eth0 address of the EC2 instance. Note that the same has to be done in 
FD_SOCK.

However, before starting docker, ports 7800 and 9000 has to be exposed, 
ie. via docker -p 7800:7800 -p 9000:9000. If more than one instances are 
started inside the docker container, ports 7801 and higher and 9001 and 
higher will also have to be exposed. This could also be done using a 
port range, e.g. -p 7800-7850:7800-7850 -p 9000-9050:9000-9050.

There are a number of ways to define external_addr:

* When starting the docker container, pass the IP address of eth0, e.g. 
docker -e EXTERNAL_ADDR=172.31.47.164. Start the JVM with 
-DEXTERNAL_ADDR=$EXTERNAL_ADDR. This will then be substituted in TCP and 
FD_SOCK.

* In the docker container (if run on AWS), you can get the private IP 
address of the host by running 'curl 
http://169.254.169.254/latest/meta-data/local-ipv4'. This could then be 
set in the JVM with -DEXTERNAL_ADDR=`curl 
http://169.254.169.254/latest/meta-data/local-ipv4`


Note that I used a fixed list of hosts in TCPPING, which is not 
something that works dynamically. I suggest used S3_PING and create an 
S3 bucket first if running on AWS for discovery.

Hope this helps,


On 09/02/17 09:29, Questions/problems related to using JGroups wrote:
> Hi, I tried to comment the  // ensureValidBindAddresses(protocols); line in
> Configurator.java I did it for version 3.6 of jgroups since version 4
> doesn't seem to work with EclipseLink (at least not with the version of
> EclipseLink that I'm using). I still get the same error message when trying
> to deploy our application:
>
> Internal Exception: Exception [EclipseLink-22118] (Eclipse Persistence
> Services - 2.6.1.v20150605-31e8258):
> org.eclipse.persistence.exceptions.RemoteCommandManagerException
> Exception Description: Failed to create JGroups connection using config
> file: /usr/local/glassfish4/glassfish/domains/domain1/config/jgroups.xml
> Internal Exception: java.net.BindException: bind_addr /172.31.24.230 is not
> a valid interface: java.net.BindException: Cannot assign requested address
> (Bind failed). Please see server.log for more details.
>
> Kind Regards
>
> Steinar
>
>
>
> --
> View this message in context: http://jgroups.1086181.n5.nabble.com/jgroups-users-Eclipselink-cache-coordination-with-JGroups-in-Docker-in-AWS-tp11257p11277.html
> Sent from the JGroups - General mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot