Re: ipyparallel: local controller / remote engines

--===============5099942430576694044==
Content-Type: multipart/alternative; boundary=001a1140fe0686491105497f6463

--001a1140fe0686491105497f6463
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 21, 2017 at 8:37 PM, jeromeku <[email protected]> wrote:

Hi,
>
> I'm trying to set up a distributed ipyparallel / ipython cluster.  I'd li=
ke
> to run the controller locally on my laptop with engines running remotely =
on
> Amazon ec2 (without resorting to Starcluster).
>
> I've read the latest documentation on setting up ssh ipython with local /
> remote hosts and tried the following:
> -Generated 'parallel' profile using "ipython create profile --parallel...=
"
> -In ipcluster_config.py:
>   -set SSHEngineSetLauncher.engines dictionary to my ec2 instance address
> and engine count, as well as set the SSHEngineLaunch.user name
> appropriately.
>   -created an alias to launch ipengine via ssh on the ec2 instance and se=
t
> the SSHEngineSetLauncher.engine_cmd accordingly.
> -In ipcontroller_config.py, set HubFactory.ip to '*' as well as in
> ipengine_config.py.
> -Set up password-less ssh login to my ec2 instance
>
> Launching ipcluster from my laptop with this configuration and setting
> log-level to debug, I see that my local ipython profile is successfully
> transferred to my ec2 instance (along with the necessary .json security
> files) and that the ipengine command is successfully executed remotely.
> However, this results in a "Registration timeout=E2=80=9D, as the ip addr=
ess in the
> transferred ipcontroller_engine.json file is set to the local (as opposed
> to
> public) IP address of my laptop.
>
> When I manually set HubFactory.ip in my ipcluster_config.py to the public
> IP
> of my laptop and re-launch ipcluster, I get a ZeroMQ error "Can't assign
> requested address.=E2=80=9D
>
> My question is, what=E2=80=99s the proper way to set the controller IP / =
location
> such so that remote engines (on AWS ec2, different LAN) can connect?
>
To connect engines to a controller over ssh you need the engines to be able
to SSH to the controller (or at least its LAN), not just the other way
around. That=E2=80=99s probably going to be trickiest bit if you want your
controller on your laptop. Technically, the connection could be done with
an ssh reverse tunnel, but IPython doesn=E2=80=99t have an option to do thi=
s for
you automatically.

In general, when engines connect over ssh, you need this config:

# ipcontroller_config.py# Tell the engines to connect over SSH:
c.IPControllerApp.engine_ssh_server =3D '[user@]host[:port]'# If engines
can only ssh to the controller's LAN, but not all the way to the
controller's host,# it must also listen on external interfaces:
c.HubFactory.ip =3D '*' # or LAN ip on the controller

It would probably be easiest, if it is acceptable, to run the controller on
the EC2 server as well. There would be a lot fewer tunnelled connections
this way. Plus, the SSHProxy launchers can work quite a bit better than
plain SSH for multiple engines on a single remote host:

# ipcluster_config.py
ssh_target =3D 'user@ec2host'

c.IPClusterStart.controller_launcher_class =3D 'SSH'
c.SSHControllerLauncher.controller_args.append('--ssh=3D' + ssh_target)
c.SSHControllerLauncher.location =3D ssh_target
c.IPClusterEngines.engine_launcher_class =3D 'SSHProxy'
c.SSHProxyEngineSetLauncher.location =3D ssh_target

-Min


> Best,
> Jerome
>
>
>
> --
> View this message in context: http://python.6.x6.nabble.com/
> ipyparallel-local-controller-remote-engines-tp5209969.html
> Sent from the IPython - User mailing list archive at Nabble.com.
> _______________________________________________
> IPython-User mailing list
> [email protected]
> https://mail.scipy.org/mailman/listinfo/ipython-user
>
=E2=80=8B

--001a1140fe0686491105497f6463
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"markdown-here-wrapper" style=3D""><p style=
=3D"margin:1.2em 0px!important">On Tue, Feb 21, 2017 at 8:37 PM, jeromeku &=
lt;<a href=3D"mailto:[email protected]" target=3D"_blank">jerome.ku@gmail=
.com</a>&gt; wrote:</p>
<p style=3D"margin:1.2em 0px!important"></p><div class=3D"markdown-here-exc=
lude"><p></p><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-=
style:solid;padding-left:1ex">Hi,<br>
<br>
I&#39;m trying to set up a distributed ipyparallel / ipython cluster.=C2=A0=
 I&#39;d like<br>
to run the controller locally on my laptop with engines running remotely on=
<br>
Amazon ec2 (without resorting to Starcluster).<br>
<br>
I&#39;ve read the latest documentation on setting up ssh ipython with local=
 /<br>
remote hosts and tried the following:<br>
-Generated &#39;parallel&#39; profile using &quot;ipython create profile --=
parallel...&quot;<br>
-In ipcluster_config.py:<br>
=C2=A0 -set SSHEngineSetLauncher.engines dictionary to my ec2 instance addr=
ess<br>
and engine count, as well as set the SSHEngineLaunch.user name<br>
appropriately.<br>
=C2=A0 -created an alias to launch ipengine via ssh on the ec2 instance and=
 set<br>
the SSHEngineSetLauncher.engine_cm<wbr>d accordingly.<br>
-In ipcontroller_config.py, set HubFactory.ip to &#39;*&#39; as well as in<=
br>
ipengine_config.py.<br>
-Set up password-less ssh login to my ec2 instance<br>
<br>
Launching ipcluster from my laptop with this configuration and setting<br>
log-level to debug, I see that my local ipython profile is successfully<br>
transferred to my ec2 instance (along with the necessary .json security<br>
files) and that the ipengine command is successfully executed remotely.<br>
However, this results in a &quot;Registration timeout=E2=80=9D, as the ip a=
ddress in the<br>
transferred ipcontroller_engine.json file is set to the local (as opposed t=
o<br>
public) IP address of my laptop.<br>
<br>
When I manually set HubFactory.ip in my ipcluster_config.py to the public I=
P<br>
of my laptop and re-launch ipcluster, I get a ZeroMQ error &quot;Can&#39;t =
assign<br>
requested address.=E2=80=9D<br>
<br>
My question is, what=E2=80=99s the proper way to set the controller IP / lo=
cation<br>
such so that remote engines (on AWS ec2, different LAN) can connect?<br></b=
lockquote><p></p></div><p style=3D"margin:1.2em 0px!important"></p>
<p style=3D"margin:1.2em 0px!important">To connect engines to a controller =
over ssh you need the engines to be able to SSH to the controller (or at le=
ast its LAN), not just the other way around. That=E2=80=99s probably going =
to be trickiest bit if you want your controller on your laptop. Technically=
, the connection could be done with an ssh reverse tunnel, but IPython does=
n=E2=80=99t have an option to do this for you automatically.</p>
<p style=3D"margin:1.2em 0px!important">In general, when engines connect ov=
er ssh, you need this config:</p>
<pre style=3D"font-size:0.85em;font-family:Consolas,Inconsolata,Courier,mon=
ospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code class=3D"hlj=
s language-python" style=3D"font-size:0.85em;font-family:Consolas,Inconsola=
ta,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wr=
ap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);bord=
er-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radi=
us:3px;border-bottom-left-radius:3px;display:inline;white-space:pre;overflo=
w:auto;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom=
-right-radius:3px;border-bottom-left-radius:3px;border:1px solid rgb(204,20=
4,204);padding:0.5em 0.7em;display:block!important;display:block;overflow-x=
:auto;padding:0.5em;color:rgb(51,51,51);background-color:rgb(248,248,248);b=
ackground-position:initial initial;background-repeat:initial initial"><span=
 class=3D"hljs-comment" style=3D"color:rgb(153,153,136);font-style:italic">=
# ipcontroller_config.py</span>
<span class=3D"hljs-comment" style=3D"color:rgb(153,153,136);font-style:ita=
lic"># Tell the engines to connect over SSH:</span>
c.IPControllerApp.engine_ssh_server =3D <span class=3D"hljs-string" style=
=3D"color:rgb(221,17,68)">&#39;[user@]host[:port]&#39;</span>
<span class=3D"hljs-comment" style=3D"color:rgb(153,153,136);font-style:ita=
lic"># If engines can only ssh to the controller&#39;s LAN, but not all the=
 way to the controller&#39;s host,</span>
<span class=3D"hljs-comment" style=3D"color:rgb(153,153,136);font-style:ita=
lic"># it must also listen on external interfaces:</span>
c.HubFactory.ip =3D <span class=3D"hljs-string" style=3D"color:rgb(221,17,6=
8)">&#39;*&#39;</span> <span class=3D"hljs-comment" style=3D"color:rgb(153,=
153,136);font-style:italic"># or LAN ip on the controller</span>
</code></pre>
<p style=3D"margin:1.2em 0px!important">It would probably be easiest, if it=
 is acceptable, to run the controller on the EC2 server as well. There woul=
d be a lot fewer tunnelled connections this way. Plus, the SSHProxy launche=
rs can work quite a bit better than plain SSH for multiple engines on a sin=
gle remote host:</p>
<pre style=3D"font-size:0.85em;font-family:Consolas,Inconsolata,Courier,mon=
ospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code class=3D"hlj=
s language-python" style=3D"font-size:0.85em;font-family:Consolas,Inconsola=
ta,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wr=
ap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);bord=
er-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radi=
us:3px;border-bottom-left-radius:3px;display:inline;white-space:pre;overflo=
w:auto;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom=
-right-radius:3px;border-bottom-left-radius:3px;border:1px solid rgb(204,20=
4,204);padding:0.5em 0.7em;display:block!important;display:block;overflow-x=
:auto;padding:0.5em;color:rgb(51,51,51);background-color:rgb(248,248,248);b=
ackground-position:initial initial;background-repeat:initial initial"><span=
 class=3D"hljs-comment" style=3D"color:rgb(153,153,136);font-style:italic">=
# ipcluster_config.py</span>
ssh_target =3D <span class=3D"hljs-string" style=3D"color:rgb(221,17,68)">&=
#39;user@ec2host&#39;</span>

c.IPClusterStart.controller_launcher_class =3D <span class=3D"hljs-string" =
style=3D"color:rgb(221,17,68)">&#39;SSH&#39;</span>
c.SSHControllerLauncher.controller_args.append(<span class=3D"hljs-string" =
style=3D"color:rgb(221,17,68)">&#39;--ssh=3D&#39;</span> + ssh_target)
c.SSHControllerLauncher.location =3D ssh_target
c.IPClusterEngines.engine_launcher_class =3D <span class=3D"hljs-string" st=
yle=3D"color:rgb(221,17,68)">&#39;SSHProxy&#39;</span>
c.SSHProxyEngineSetLauncher.location =3D ssh_target
</code></pre>
<p style=3D"margin:1.2em 0px!important">-Min</p>
<p style=3D"margin:1.2em 0px!important"></p><div class=3D"markdown-here-exc=
lude"><p></p><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-=
style:solid;padding-left:1ex">
<br>
Best,<br>
Jerome<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://python.6.x6.nabble.com/ipyp=
arallel-local-controller-remote-engines-tp5209969.html" rel=3D"noreferrer" =
target=3D"_blank">http://python.6.x6.nabble.com/<wbr>ipyparallel-local-cont=
roller-r<wbr>emote-engines-tp5209969.html</a><br>
Sent from the IPython - User mailing list archive at Nabble.com.<br>
______________________________<wbr>_________________<br>
IPython-User mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">IPython-User@sc=
ipy.org</a><br>
<a href=3D"https://mail.scipy.org/mailman/listinfo/ipython-user" rel=3D"nor=
eferrer" target=3D"_blank">https://mail.scipy.org/mailman<wbr>/listinfo/ipy=
thon-user</a><br>
</blockquote><p></p></div><p style=3D"margin:1.2em 0px!important"></p>
<div title=3D"MDH:PGJyPjxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj48YnI+PGRpdiBjbGFzc=
z0iZ21haWxfcXVvdGUi
Pk9uIFR1ZSwgRmViIDIxLCAyMDE3IGF0IDg6MzcgUE0sIGplcm9tZWt1IDxzcGFuIGRpcj0ibHR=
y
Ij4mbHQ7PGEgaHJlZj0ibWFpbHRvOmplcm9tZS5rdUBnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5=
r
Ij5qZXJvbWUua3VAZ21haWwuY29tPC9hPiZndDs8L3NwYW4+IHdyb3RlOjxicj48YmxvY2txdW9=
0
ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46IDBweCAwcHggMHB4IDAuOGV4OyB=
i
b3JkZXItbGVmdC13aWR0aDogMXB4OyBib3JkZXItbGVmdC1jb2xvcjogcmdiKDIwNCwgMjA0LCA=
y
MDQpOyBib3JkZXItbGVmdC1zdHlsZTogc29saWQ7IHBhZGRpbmctbGVmdDogMWV4OyI+SGksPGJ=
y
Pgo8YnI+CkknbSB0cnlpbmcgdG8gc2V0IHVwIGEgZGlzdHJpYnV0ZWQgaXB5cGFyYWxsZWwgLyB=
p
cHl0aG9uIGNsdXN0ZXIuJm5ic3A7IEknZCBsaWtlPGJyPgp0byBydW4gdGhlIGNvbnRyb2xsZXI=
g
bG9jYWxseSBvbiBteSBsYXB0b3Agd2l0aCBlbmdpbmVzIHJ1bm5pbmcgcmVtb3RlbHkgb248YnI=
+
CkFtYXpvbiBlYzIgKHdpdGhvdXQgcmVzb3J0aW5nIHRvIFN0YXJjbHVzdGVyKS48YnI+Cjxicj4=
K
SSd2ZSByZWFkIHRoZSBsYXRlc3QgZG9jdW1lbnRhdGlvbiBvbiBzZXR0aW5nIHVwIHNzaCBpcHl=
0
aG9uIHdpdGggbG9jYWwgLzxicj4KcmVtb3RlIGhvc3RzIGFuZCB0cmllZCB0aGUgZm9sbG93aW5=
n
Ojxicj4KLUdlbmVyYXRlZCAncGFyYWxsZWwnIHByb2ZpbGUgdXNpbmcgImlweXRob24gY3JlYXR=
l
IHByb2ZpbGUgLS1wYXJhbGxlbC4uLiI8YnI+Ci1JbiBpcGNsdXN0ZXJfY29uZmlnLnB5Ojxicj4=
K
Jm5ic3A7IC1zZXQgU1NIRW5naW5lU2V0TGF1bmNoZXIuZW5naW5lcyBkaWN0aW9uYXJ5IHRvIG1=
5
IGVjMiBpbnN0YW5jZSBhZGRyZXNzPGJyPgphbmQgZW5naW5lIGNvdW50LCBhcyB3ZWxsIGFzIHN=
l
dCB0aGUgU1NIRW5naW5lTGF1bmNoLnVzZXIgbmFtZTxicj4KYXBwcm9wcmlhdGVseS48YnI+CiZ=
u
YnNwOyAtY3JlYXRlZCBhbiBhbGlhcyB0byBsYXVuY2ggaXBlbmdpbmUgdmlhIHNzaCBvbiB0aGU=
g
ZWMyIGluc3RhbmNlIGFuZCBzZXQ8YnI+CnRoZSBTU0hFbmdpbmVTZXRMYXVuY2hlci5lbmdpbmV=
f
Y208d2JyPmQgYWNjb3JkaW5nbHkuPGJyPgotSW4gaXBjb250cm9sbGVyX2NvbmZpZy5weSwgc2V=
0
IEh1YkZhY3RvcnkuaXAgdG8gJyonIGFzIHdlbGwgYXMgaW48YnI+CmlwZW5naW5lX2NvbmZpZy5=
w
eS48YnI+Ci1TZXQgdXAgcGFzc3dvcmQtbGVzcyBzc2ggbG9naW4gdG8gbXkgZWMyIGluc3RhbmN=
l
PGJyPgo8YnI+CkxhdW5jaGluZyBpcGNsdXN0ZXIgZnJvbSBteSBsYXB0b3Agd2l0aCB0aGlzIGN=
v
bmZpZ3VyYXRpb24gYW5kIHNldHRpbmc8YnI+CmxvZy1sZXZlbCB0byBkZWJ1ZywgSSBzZWUgdGh=
h
dCBteSBsb2NhbCBpcHl0aG9uIHByb2ZpbGUgaXMgc3VjY2Vzc2Z1bGx5PGJyPgp0cmFuc2ZlcnJ=
l
ZCB0byBteSBlYzIgaW5zdGFuY2UgKGFsb25nIHdpdGggdGhlIG5lY2Vzc2FyeSAuanNvbiBzZWN=
1
cml0eTxicj4KZmlsZXMpIGFuZCB0aGF0IHRoZSBpcGVuZ2luZSBjb21tYW5kIGlzIHN1Y2Nlc3N=
m
dWxseSBleGVjdXRlZCByZW1vdGVseS48YnI+Ckhvd2V2ZXIsIHRoaXMgcmVzdWx0cyBpbiBhICJ=
S
ZWdpc3RyYXRpb24gdGltZW91dOKAnSwgYXMgdGhlIGlwIGFkZHJlc3MgaW4gdGhlPGJyPgp0cmF=
u
c2ZlcnJlZCBpcGNvbnRyb2xsZXJfZW5naW5lLmpzb24gZmlsZSBpcyBzZXQgdG8gdGhlIGxvY2F=
s
IChhcyBvcHBvc2VkIHRvPGJyPgpwdWJsaWMpIElQIGFkZHJlc3Mgb2YgbXkgbGFwdG9wLjxicj4=
K
PGJyPgpXaGVuIEkgbWFudWFsbHkgc2V0IEh1YkZhY3RvcnkuaXAgaW4gbXkgaXBjbHVzdGVyX2N=
v
bmZpZy5weSB0byB0aGUgcHVibGljIElQPGJyPgpvZiBteSBsYXB0b3AgYW5kIHJlLWxhdW5jaCB=
p
cGNsdXN0ZXIsIEkgZ2V0IGEgWmVyb01RIGVycm9yICJDYW4ndCBhc3NpZ248YnI+CnJlcXVlc3R=
l
ZCBhZGRyZXNzLuKAnTxicj4KPGJyPgpNeSBxdWVzdGlvbiBpcywgd2hhdOKAmXMgdGhlIHByb3B=
l
ciB3YXkgdG8gc2V0IHRoZSBjb250cm9sbGVyIElQIC8gbG9jYXRpb248YnI+CnN1Y2ggc28gdGh=
h
dCByZW1vdGUgZW5naW5lcyAob24gQVdTIGVjMiwgZGlmZmVyZW50IExBTikgY2FuIGNvbm5lY3Q=
/
PGJyPjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PlRvIGNvbm5lY3QgZW5naW5lcyB=
0
byBhIGNvbnRyb2xsZXIgb3ZlciBzc2ggeW91IG5lZWQgdGhlIGVuZ2luZXMgdG8gYmUgYWJsZSB=
0
byBTU0ggdG8gdGhlIGNvbnRyb2xsZXIgKG9yIGF0IGxlYXN0IGl0cyBMQU4pLCBub3QganVzdCB=
0
aGUgb3RoZXIgd2F5IGFyb3VuZC4gVGhhdCdzIHByb2JhYmx5IGdvaW5nIHRvIGJlIHRyaWNraWV=
z
dCBiaXQgaWYgeW91IHdhbnQgeW91ciBjb250cm9sbGVyIG9uIHlvdXIgbGFwdG9wLiBUZWNobml=
j
YWxseSwgdGhlIGNvbm5lY3Rpb24gY291bGQgYmUgZG9uZSB3aXRoIGFuIHNzaCByZXZlcnNlIHR=
1
bm5lbCwgYnV0IElQeXRob24gZG9lc24ndCBoYXZlIGFuIG9wdGlvbiB0byBkbyB0aGlzIGZvciB=
5
b3UgYXV0b21hdGljYWxseS48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkluIGdlbmVyYWwsIHd=
o
ZW4gZW5naW5lcyBjb25uZWN0IG92ZXIgc3NoLCB5b3UgbmVlZCB0aGlzIGNvbmZpZzo8L2Rpdj4=
8
ZGl2Pjxicj48L2Rpdj48ZGl2PmBgYHB5dGhvbjwvZGl2PjxkaXY+IyBpcGNvbnRyb2xsZXJfY29=
u
ZmlnLnB5PC9kaXY+PGRpdj4jIFRlbGwgdGhlIGVuZ2luZXMgdG8gY29ubmVjdCBvdmVyIFNTSDo=
8
L2Rpdj48ZGl2PmMuSVBDb250cm9sbGVyQXBwLmVuZ2luZV9zc2hfc2VydmVyID0gJ1t1c2VyQF1=
o
b3N0Wzpwb3J0XSc8L2Rpdj48ZGl2PiMgSWYgZW5naW5lcyBjYW4gb25seSBzc2ggdG8gdGhlIGN=
v
bnRyb2xsZXIncyBMQU4sIGJ1dCBub3QgYWxsIHRoZSB3YXkgdG8gdGhlIGNvbnRyb2xsZXIncyB=
o
b3N0LDwvZGl2PjxkaXY+IyBpdCBtdXN0IGFsc28gbGlzdGVuIG9uIGV4dGVybmFsIGludGVyZmF=
j
ZXM6PC9kaXY+PGRpdj5jLkh1YkZhY3RvcnkuaXAgPSAnKicgIyBvciBMQU4gaXAgb24gdGhlIGN=
v
bnRyb2xsZXI8YnI+PC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pkl0IHd=
v
dWxkIHByb2JhYmx5IGJlIGVhc2llc3QsIGlmIGl0IGlzIGFjY2VwdGFibGUsIHRvIHJ1biB0aGU=
g
Y29udHJvbGxlciBvbiB0aGUgRUMyIHNlcnZlciBhcyB3ZWxsLiBUaGVyZSB3b3VsZCBiZSBhIGx=
v
dCBmZXdlciB0dW5uZWxsZWQgY29ubmVjdGlvbnMgdGhpcyB3YXkuIFBsdXMsIHRoZSBTU0hQcm9=
4
eSBsYXVuY2hlcnMgY2FuIHdvcmsgcXVpdGUgYSBiaXQgYmV0dGVyIHRoYW4gcGxhaW4gU1NIIGZ=
v
ciBtdWx0aXBsZSBlbmdpbmVzIG9uIGEgc2luZ2xlIHJlbW90ZSBob3N0OjwvZGl2PjxkaXY+PGJ=
y
PjwvZGl2PjxkaXY+YGBgcHl0aG9uPC9kaXY+PGRpdj4jIGlwY2x1c3Rlcl9jb25maWcucHk8L2R=
p
dj48ZGl2PnNzaF90YXJnZXQgPSAndXNlckBlYzJob3N0JzwvZGl2PjxkaXY+PGJyPjwvZGl2Pjx=
k
aXY+Yy5JUENsdXN0ZXJTdGFydC5jb250cm9sbGVyX2xhdW5jaGVyX2NsYXNzID0gJ1NTSCc8L2R=
p
dj48ZGl2PmMuU1NIQ29udHJvbGxlckxhdW5jaGVyLmNvbnRyb2xsZXJfYXJncy5hcHBlbmQoJy0=
t
c3NoPScgKyBzc2hfdGFyZ2V0KTwvZGl2PjxkaXY+Yy5TU0hDb250cm9sbGVyTGF1bmNoZXIubG9=
j
YXRpb24gPSBzc2hfdGFyZ2V0PC9kaXY+PGRpdj5jLklQQ2x1c3RlckVuZ2luZXMuZW5naW5lX2x=
h
dW5jaGVyX2NsYXNzID0gJ1NTSFByb3h5JzwvZGl2PjxkaXY+Yy5TU0hQcm94eUVuZ2luZVNldEx=
h
dW5jaGVyLmxvY2F0aW9uID0gc3NoX3RhcmdldDwvZGl2PjxkaXY+YGBgPC9kaXY+PGRpdj48YnI=
+
PC9kaXY+PGRpdj4tTWluPC9kaXY+PGRpdj4mbmJzcDs8L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0=
i
Z21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46IDBweCAwcHggMHB4IDAuOGV4OyBib3JkZXItbGV=
m
dC13aWR0aDogMXB4OyBib3JkZXItbGVmdC1jb2xvcjogcmdiKDIwNCwgMjA0LCAyMDQpOyBib3J=
k
ZXItbGVmdC1zdHlsZTogc29saWQ7IHBhZGRpbmctbGVmdDogMWV4OyI+Cjxicj4KQmVzdCw8YnI=
+
Ckplcm9tZTxicj4KPGJyPgo8YnI+Cjxicj4KLS08YnI+ClZpZXcgdGhpcyBtZXNzYWdlIGluIGN=
v
bnRleHQ6IDxhIGhyZWY9Imh0dHA6Ly9weXRob24uNi54Ni5uYWJibGUuY29tL2lweXBhcmFsbGV=
s
LWxvY2FsLWNvbnRyb2xsZXItcmVtb3RlLWVuZ2luZXMtdHA1MjA5OTY5Lmh0bWwiIHJlbD0ibm9=
y
ZWZlcnJlciIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9weXRob24uNi54Ni5uYWJibGUuY29tLzx=
3
YnI+aXB5cGFyYWxsZWwtbG9jYWwtY29udHJvbGxlci1yPHdicj5lbW90ZS1lbmdpbmVzLXRwNTI=
w
OTk2OS5odG1sPC9hPjxicj4KU2VudCBmcm9tIHRoZSBJUHl0aG9uIC0gVXNlciBtYWlsaW5nIGx=
p
c3QgYXJjaGl2ZSBhdCBOYWJibGUuY29tLjxicj4KX19fX19fX19fX19fX19fX19fX19fX19fX19=
f
X19fPHdicj5fX19fX19fX19fX19fX19fXzxicj4KSVB5dGhvbi1Vc2VyIG1haWxpbmcgbGlzdDx=
i
cj4KPGEgaHJlZj0ibWFpbHRvOklQeXRob24tVXNlckBzY2lweS5vcmciIHRhcmdldD0iX2JsYW5=
r
Ij5JUHl0aG9uLVVzZXJAc2NpcHkub3JnPC9hPjxicj4KPGEgaHJlZj0iaHR0cHM6Ly9tYWlsLnN=
j
aXB5Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lweXRob24tdXNlciIgcmVsPSJub3JlZmVycmVyIiB=
0
YXJnZXQ9Il9ibGFuayI+aHR0cHM6Ly9tYWlsLnNjaXB5Lm9yZy9tYWlsbWFuPHdicj4vbGlzdGl=
u
Zm8vaXB5dGhvbi11c2VyPC9hPjxicj4KPC9ibG9ja3F1b3RlPjwvZGl2Pjxicj48L2Rpdj4=3D"=
 style=3D"height:0;width:0;max-height:0;max-width:0;overflow:hidden;font-si=
ze:0em;padding:0;margin:0">=E2=80=8B</div></div></div>

--001a1140fe0686491105497f6463--

--===============5099942430576694044==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
IPython-User mailing list
[email protected]
https://mail.scipy.org/mailman/listinfo/ipython-user

--===============5099942430576694044==--