RE: ipv6 support on cayenne_1_5_0_restricted

John,

We recently implemented the new network services API
natively on Linux. Previously we had implemented the
new network services API on Linux using the old 
network services API. I would encourage you to play
with this and test it. The developer who implemented it
has dev-tested it, but it has not gone through a formal
QA cycle.

Currently this is only checked into the HEAD.

Eric

=======================================
Eric Hyche ([email protected])
Principal Engineer
RealNetworks, Inc.


>-----Original Message-----
>From: [email protected] [mailto:[email protected]]
>On Behalf Of John Stirling
>Sent: Friday, October 17, 2008 6:33 AM
>To: [email protected]
>Subject: Re: [Helix-client-dev] ipv6 support on cayenne_1_5_0_restricted
>
>What's the latest status on helix ipv6 support on arm-linux ?
>
>If supported, which branch do we need ?
>
>John
>
>
>milko wrote:
>>
>> There are no IPV6 plans for Unix for the next 6 months in Real.
>> The amount of work needed to get IPV6 working on Linux was estimated
>> as follows for somebody not familiar with this code but familiar with
>> Helix:
>>
>> IPV6 Network stack integration - Select 7 days
>> IPV6 Network stack integration - Socket EventPending marshalling 5 days
>> IPV6 Network stack integration - Resolver 3 days
>>
>> It is quite possible that use of network thread is not necessary at
>> all. I'd suggest turning off network thread for initial implementation.
>>
>> I am including some past observations while scoping the work needed
>> for Linux.
>> These are not recent however and should used as an introduction rather
>> than a blue print.
>> Liam Murray is most familiar with this code and might be able to
>> answer specific questions on this.
>>
>> 7/23/04:
>> -------------------------------------------------------------
>> Needed:
>> 1.)
>> HXSocketSelector class provided in common/netio/platform/win should be
>> given a cross platform base class for its cross platform functionality
>> or at least the bridge class to actual platform specific implementations.
>>
>> 2.)
>> HXSocketSelector class should be given a new static method
>> Select(UINT32 ulTimeout), wehere ulTimeout of 0 indicates polling and
>> timeout of 0xFFFFFFFF indicates blocking call (WAIT_NEVER and
>> WAIT_FOREVER).
>> This method needs to be implemented for unix and do nothing on windows
>> (base class default behavior).
>>
>> 3.)
>> HXSocketSelector::Select(ulPollingInterval) needs to be implemented in
>> the main loop of appropriate applications.
>>
>> 9/9/04: (From Liam Murray)
>> -----------------------------------------------------------------
>> For the record, here is what I know regarding thread message passing
>> and issues relevant to making the new socket/select/resolver
>> implementation work on non-Windows platforms.
>> 1) HXThread::MakeThread is a static "factory" method that creates a
>> platform-specific instance of HXThread (e.g., new HXWinThread on
>> widows). Then you call CreateThread() on the instanciated thread
>> object to create the actual system thread. (There is code all over the
>> place wherever thread objects are created that usesTHREADS_SUPPORTED
>> to determine which creation function, MakeThread() or MakeStubThread()
>> to call. I'm not sure why the logic to decide whether to create a stub
>> vs. real thread is not encapsulated in MakeThread() itself.)
>>
>> 2) On Windows you get the HXThread for the thread you want to send the
>> message to and you call PostMessage(). That actually calls the
>> underlying WinAPI ::PostMessage(), so the message ends up in the
>> target thread's system (os maintained) message queue.
>> Similarly on Windows when you call HXThread::GetMessage() and
>> HXThread::DispatchMessage() these go through to WinAPI methods of the
>> same name. GetMessage() pulls the message from the thread's os
>> maintained message queue. (There is a similar function named
>> PeekMessage(), so anywhere I refer to GetMessage(), PeekMessage() can
>> be substituted or may in fact be the function that is called.)
>> On Unix, unlike on Windows, no system thread message queue or thread
>> message service is used. That is, HXThread creates its own message
>> queue. When you call HXThread::PostMessage() the message is added to
>> that queue (guarded by a mutex). When you call HXThread::GetMessage()
>> the reverse is done; the message at the front of the queue is popped off.
>>
>> 3) On windows when you call the Win API PostMessage() (via
>> HXThread::PostMessage()) you can pass an HWND as a parameter to the
>> function. Each HWND has a WinProc associated with it. When a thread's
>> message loop calls DispatchMessage() (after retrieving the message
>> with GetMessage()), the WinProc associated with the hwnd specified in
>> the message is called.
>> Note that on Windows typically the top level UI code has a message
>> loop running on the main app thread. (GetMessage, DispatchMessage,
>> etc.) Also not that HXThread::GetMessage() picks up all thread
>> messages, including those (such as WM_TIMER, e.g.) posted to the
>> thread directly via Win API.
>> HXThread::DispatchMessage() is not implemented on Unix
>>
>> 4) In the original network code the network thread needed to send
>> messages to the app thread (more specifically, the thread on which
>> CreateEngine() is called to create the HXClientEngine()).
>>
>> 5) On Windows when the net thread is created a dummy invisible window
>> is created. A WinProc is associated with that window. When the net
>> thread needs to send a message to the main app thread it calls
>> HXThread::PostMessage and passes the dummy window handle. The TLC
>> message loop gets the message, then calls DispatchMessage, which calls
>> the dummy window WinProc. (The WinProc is
>> CAsyncNetThread::AsyncNotifyProc() in common/netio .) The WinProc then
>> forwards calls to ThreadedConn member functions.
>> The nice thing about this is that the TLC does not need to know what
>> message handlers are doing. We can write a function (a WinProc) in
>> some module and then associated that with a window, and we can hook
>> into messages being sent to that thread automatically with no
>> coordination with the message loop itself needed.
>>
>> 6) On Unix, there is no UI message loop calling the equivalent of
>> DispatchMessage(). There is no equivalent of an HWND and associated
>> WinProc that can handle thread messages. That is where EventOccurred()
>> comes in. EventOccurred() (called in response to a timer click) calls
>> GetMessage() and for the most part asssumes they are messages from the
>> network thread. It then calls member functions of ThreadedConn.
>>
>> 7) On Windows when you call WSAAsyncSelect() you specify an HWND for
>> the system to use when posting select noticifications. When the system
>> detects a change in socket state it posts a message (with the HWND
>> passed in the WSAAsyncSelect() specified) to the thread message queue
>> for the thread that called WSAAsyncSelect(). The message ends up being
>> sent to the WinProc as described in (5) above. In the old net code
>> another dummy window was created solely for receiving select messages
>>
>> 8) In the old net thread implementation the net thread continually
>> calls HXThread::GetMessage(). Since timer messages are posted
>> regularly to the net thread it wakes up periodically even if there are
>> no network messages. On Unix unix_net::process_idle() is called after
>> GetMessage() returns. At this point the Unix implementation calls
>> ::select() (with a short timeout) to poll the state of the sockets.
>>
>> When I did the new client net API implementation I wanted to
>> encapsulate and isolate the message passing, receiving and socket
>> select platform differences as much as possible. That way the socket
>> implementation, the resolving and the net thread code could be as free
>> of platform dependencies and #ifdefs as much as possible. That is what
>> HXThreadMessageSink and HXSocketSelector are for. (Ideally none of the
>> other new socket api implementation will need updating/porting for
>> non-Windows platforms, only these classes.)
>> For Unix HXSocketSelector will need a function like
>> "HXSockeSelector::Process()" akin to unix_net::process_idle(). This
>> will need to be called in the thread message loop or in a timer
>> function. Process() would probably be implemented very similarly to
>> unix_net::process_idle(). Other code that is currently using
>> HXSocketSelector (i.e., hxclientsockimp) should then "just work".
>> HXThreadMessageSink basically maintains a list of callback handlers
>> for thread messages. If you want to handle a certain thread message
>> you look up the thread message sink and register a callback associated
>> with the message. That is what the threaded socket code does. It
>> registers to handle messages that the net thread posts back to the app
>> thread. The resolver registers to receive messages that are posted
>> back from the resolver thread(s). There is no dependency between the
>> resolver, threaded socket and thread message dispatcher other than in
>> a generic way in terms of participation in this thread message passing
>> scheme. For example, if some other future code needs to pass messages
>> between threads, it only needs to define a message and register a
>> handler for the message with the thread message sink.
>> Each thread message sink is associated with a unique handle. On
>> windows the handle is actually an HWND. In fact, HXThreadMessageSink
>> creates a dummy hwnd for hooking into the TLC-owned message loop. (On
>> other platfroms the ID can most likely be 0.)
>> When we pass a message to the main app thread (from net thread to
>> client socket code running on app thread) we specify the handle of the
>> thread message sink associated with the app thread in PostMessage().
>> Because PostMessage() goes through the underlying WinAPI, the message
>> ends up in the window associated with HXThreadMessageSink() and that
>> is then forwarded to the handler for the message. The handler is
>> registered at some point during initialization by the network code.
>> In the case of calling WSAAsyncSelect() on Windows we lookup the
>> thread message sink for the thread on which we want to receive select
>> notifications and pass the HWND (the sink's ID) to WSAAsyncSelect.
>> (First we register a callback to handle the select messages sent to
>> the thread.)
>> On non-windows platforms I was thinking that EventOccurred (or in the
>> case of a non-main-app thread, the thread message loop that calls
>> GetMessage()) would just forward messages through the thread message
>> sink (which would call the appropriate handler for the message).
>> Perhaps HXThread::DispatchMessage() on Unix (now not implemented)
>> could just look up the message sink associated with its thread and
>> pass the message on to the thread message sink to forward to the
>> handler registered to handle the message. (HXThreadMessageSink could
>> be hidden behind or part of HXThread even.)
>> The TLS stuff is intended to ensure that if you call select on one
>> thread (via HXSocketSelector) you get select notifications back on the
>> same thread. It is also used to look up the HXThreadMessageSink
>> per-thread instance associated with each thread. We can work around
>> this by just keying on a thread id to associate thread-specific data.
>> (In the old net implementation there was one select being done on one
>> thread and these results were posted back to the main app thread. If
>> we can assume select will only ever be needed/called on one thread
>> then we do not need the TLS at all.)
>> None of this is set in stone. As I said my primary goal was to
>> encapsulate the stuff that differed significantly between platforms
>> (thread message passing and select) and get things working on Windows,
>> while at the same time trying to keep in mind eventual non-Windows
>> platform needs.
>>
>>
>>
>> At 10:48 AM 5/24/2005, Nicholas Hart wrote:
>>> I was hoping that person would step up... :)
>>>
>>> Milko? Henry? Greg?
>>>
>>>
>>> On Tue, 2005-05-24 at 06:23, John Stirling wrote:
>>> > Any idea who the person to ask is ?
>>> >
>>> >
>>> > On Mon, 2005-05-23 at 21:29, Nicholas Hart wrote:
>>> > > I think someone else on this list should have more info than I about
>>> > > IPV6 plans for linux. I think there is more to do than just
>>> > > implementing some functionality in the threading code (that part is
>>> > > mostly just hooking up the helix APIs to the pthread APIs--not too
>>> > > tricky).
>>> > >
>>> > >
>>> > > On Mon, 2005-05-23 at 08:46, Philip Blundell wrote:
>>> > > > That does seem to be the case, yeah; HELIX_CLIENT_IPV6_NET_API
>>> is only
>>> > > > enabled #ifdef _WINDOWS. I tried switching this on for linux as
>>> well,
>>> > > > but the resulting splay blows up at this code in
>>> > > > hxthreadmessagesink.cpp:
>>> > > >
>>> > > > #if defined(_WINDOWS)
>>> > > > // create instance for this thread
>>> > > > hr = HXThreadMessageSink::Create(pStored);
>>> > > > #else
>>> > > > HX_ASSERT(false);
>>> > > > hr = HXR_NOTIMPL;
>>> > > > #endif
>>> > > >
>>> > > > which is called from
>>> CHXNetThreadClientNetServices::DoDriverInit. I
>>> > > > didn't understand enough about the threading code to figure out
>>> what
>>> > > > needs to be done to make all this work.
>>> > > >
>>> > > > p.
>>> > > >
>>> > > > On Thu, 2005-05-19 at 10:26 -0700, Nicholas Hart wrote:
>>> > > > > It's my understanding that IPV6 is not yet working on linux.
>>> There
>>> > > > > was an overhaul of the networking layer for cayenne, which
>>> includes
>>> > > > > IPV6, but due to resource limitations I believe it was only
>>> fully
>>> > > > > implemented on Windows. I'm not up-to-date on our current
>>> plans, so
>>> > > > > maybe someone else on this list can provide some more help.
>>> > > > >
>>> > > > >
>>> > > > > On May 19, 2005, at 8:51 AM, John Stirling wrote:
>>> > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > Is IPv6 expected to work on cayenne_1_5_0 (i586 or
>>> arm-linux) ?
>>> > > > > >
>>> > > > > > I've just done a clean (i586) build using : build -m
>>> > > > > > hxclient_1_5_0_cayenne_restricted -P
>>> helix-client-all-defines -
>>> > > > > > trelease
>>> > > > > > -D 20050518 splay
>>> > > > > > which seems to produce all the expected .so files
>>> > > > > >
>>> > > > > > We can play IPv4 streams ok with this build but can't get a
>>> sample
>>> > > > > > IPv6
>>> > > > > > stream to work : http://virgin.6pack.org/vruk-lo-mp3
>>> > > > > >
>>> > > > > > John
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > _______________________________________________
>>> > > > > > Helix-client-dev mailing list
>>> > > > > > [email protected]
>>> > > > > >
>>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > > > _______________________________________________
>>> > > > > Helix-client-dev mailing list
>>> > > > > [email protected]
>>> > > > >
>>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>> --
>>> Nicholas Hart
>>> [email protected]
>>> Technical Lead, Helix Player
>>> https://player.helixcommunity.org
>>> http://www.real.com
>>>
>>>
>>>
>>> _______________________________________________
>>> Helix-client-dev mailing list
>>> [email protected]
>>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>
>>
>
>
>_______________________________________________
>Helix-client-dev mailing list
>[email protected]
>http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev


_______________________________________________
Helix-client-dev mailing list
[email protected]
http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev