Re: ipv6 support on cayenne_1_5_0_restricted

Eric,

OK, we'll keep that in mind.

I presume the changes are at least partly in clntcore.so ? If not, what 
plugins are affected ? It's not just a case of being able to take one 
plugin from HEAD and use it on a cayenne build ?

When do you plan to create a stable branch off HEAD again ?

John


Eric Hyche wrote:
> John,
>
> We recently implemented the new network services API
> natively on Linux. Previously we had implemented the
> new network services API on Linux using the old 
> network services API. I would encourage you to play
> with this and test it. The developer who implemented it
> has dev-tested it, but it has not gone through a formal
> QA cycle.
>
> Currently this is only checked into the HEAD.
>
> Eric
>
> =======================================
> Eric Hyche ([email protected])
> Principal Engineer
> RealNetworks, Inc.
>
>
>   
>> -----Original Message-----
>> From: [email protected] [mailto:[email protected]]
>> On Behalf Of John Stirling
>> Sent: Friday, October 17, 2008 6:33 AM
>> To: [email protected]
>> Subject: Re: [Helix-client-dev] ipv6 support on cayenne_1_5_0_restricted
>>
>> What's the latest status on helix ipv6 support on arm-linux ?
>>
>> If supported, which branch do we need ?
>>
>> John
>>
>>
>> milko wrote:
>>     
>>> There are no IPV6 plans for Unix for the next 6 months in Real.
>>> The amount of work needed to get IPV6 working on Linux was estimated
>>> as follows for somebody not familiar with this code but familiar with
>>> Helix:
>>>
>>> IPV6 Network stack integration - Select 7 days
>>> IPV6 Network stack integration - Socket EventPending marshalling 5 days
>>> IPV6 Network stack integration - Resolver 3 days
>>>
>>> It is quite possible that use of network thread is not necessary at
>>> all. I'd suggest turning off network thread for initial implementation.
>>>
>>> I am including some past observations while scoping the work needed
>>> for Linux.
>>> These are not recent however and should used as an introduction rather
>>> than a blue print.
>>> Liam Murray is most familiar with this code and might be able to
>>> answer specific questions on this.
>>>
>>> 7/23/04:
>>> -------------------------------------------------------------
>>> Needed:
>>> 1.)
>>> HXSocketSelector class provided in common/netio/platform/win should be
>>> given a cross platform base class for its cross platform functionality
>>> or at least the bridge class to actual platform specific implementations.
>>>
>>> 2.)
>>> HXSocketSelector class should be given a new static method
>>> Select(UINT32 ulTimeout), wehere ulTimeout of 0 indicates polling and
>>> timeout of 0xFFFFFFFF indicates blocking call (WAIT_NEVER and
>>> WAIT_FOREVER).
>>> This method needs to be implemented for unix and do nothing on windows
>>> (base class default behavior).
>>>
>>> 3.)
>>> HXSocketSelector::Select(ulPollingInterval) needs to be implemented in
>>> the main loop of appropriate applications.
>>>
>>> 9/9/04: (From Liam Murray)
>>> -----------------------------------------------------------------
>>> For the record, here is what I know regarding thread message passing
>>> and issues relevant to making the new socket/select/resolver
>>> implementation work on non-Windows platforms.
>>> 1) HXThread::MakeThread is a static "factory" method that creates a
>>> platform-specific instance of HXThread (e.g., new HXWinThread on
>>> widows). Then you call CreateThread() on the instanciated thread
>>> object to create the actual system thread. (There is code all over the
>>> place wherever thread objects are created that usesTHREADS_SUPPORTED
>>> to determine which creation function, MakeThread() or MakeStubThread()
>>> to call. I'm not sure why the logic to decide whether to create a stub
>>> vs. real thread is not encapsulated in MakeThread() itself.)
>>>
>>> 2) On Windows you get the HXThread for the thread you want to send the
>>> message to and you call PostMessage(). That actually calls the
>>> underlying WinAPI ::PostMessage(), so the message ends up in the
>>> target thread's system (os maintained) message queue.
>>> Similarly on Windows when you call HXThread::GetMessage() and
>>> HXThread::DispatchMessage() these go through to WinAPI methods of the
>>> same name. GetMessage() pulls the message from the thread's os
>>> maintained message queue. (There is a similar function named
>>> PeekMessage(), so anywhere I refer to GetMessage(), PeekMessage() can
>>> be substituted or may in fact be the function that is called.)
>>> On Unix, unlike on Windows, no system thread message queue or thread
>>> message service is used. That is, HXThread creates its own message
>>> queue. When you call HXThread::PostMessage() the message is added to
>>> that queue (guarded by a mutex). When you call HXThread::GetMessage()
>>> the reverse is done; the message at the front of the queue is popped off.
>>>
>>> 3) On windows when you call the Win API PostMessage() (via
>>> HXThread::PostMessage()) you can pass an HWND as a parameter to the
>>> function. Each HWND has a WinProc associated with it. When a thread's
>>> message loop calls DispatchMessage() (after retrieving the message
>>> with GetMessage()), the WinProc associated with the hwnd specified in
>>> the message is called.
>>> Note that on Windows typically the top level UI code has a message
>>> loop running on the main app thread. (GetMessage, DispatchMessage,
>>> etc.) Also not that HXThread::GetMessage() picks up all thread
>>> messages, including those (such as WM_TIMER, e.g.) posted to the
>>> thread directly via Win API.
>>> HXThread::DispatchMessage() is not implemented on Unix
>>>
>>> 4) In the original network code the network thread needed to send
>>> messages to the app thread (more specifically, the thread on which
>>> CreateEngine() is called to create the HXClientEngine()).
>>>
>>> 5) On Windows when the net thread is created a dummy invisible window
>>> is created. A WinProc is associated with that window. When the net
>>> thread needs to send a message to the main app thread it calls
>>> HXThread::PostMessage and passes the dummy window handle. The TLC
>>> message loop gets the message, then calls DispatchMessage, which calls
>>> the dummy window WinProc. (The WinProc is
>>> CAsyncNetThread::AsyncNotifyProc() in common/netio .) The WinProc then
>>> forwards calls to ThreadedConn member functions.
>>> The nice thing about this is that the TLC does not need to know what
>>> message handlers are doing. We can write a function (a WinProc) in
>>> some module and then associated that with a window, and we can hook
>>> into messages being sent to that thread automatically with no
>>> coordination with the message loop itself needed.
>>>
>>> 6) On Unix, there is no UI message loop calling the equivalent of
>>> DispatchMessage(). There is no equivalent of an HWND and associated
>>> WinProc that can handle thread messages. That is where EventOccurred()
>>> comes in. EventOccurred() (called in response to a timer click) calls
>>> GetMessage() and for the most part asssumes they are messages from the
>>> network thread. It then calls member functions of ThreadedConn.
>>>
>>> 7) On Windows when you call WSAAsyncSelect() you specify an HWND for
>>> the system to use when posting select noticifications. When the system
>>> detects a change in socket state it posts a message (with the HWND
>>> passed in the WSAAsyncSelect() specified) to the thread message queue
>>> for the thread that called WSAAsyncSelect(). The message ends up being
>>> sent to the WinProc as described in (5) above. In the old net code
>>> another dummy window was created solely for receiving select messages
>>>
>>> 8) In the old net thread implementation the net thread continually
>>> calls HXThread::GetMessage(). Since timer messages are posted
>>> regularly to the net thread it wakes up periodically even if there are
>>> no network messages. On Unix unix_net::process_idle() is called after
>>> GetMessage() returns. At this point the Unix implementation calls
>>> ::select() (with a short timeout) to poll the state of the sockets.
>>>
>>> When I did the new client net API implementation I wanted to
>>> encapsulate and isolate the message passing, receiving and socket
>>> select platform differences as much as possible. That way the socket
>>> implementation, the resolving and the net thread code could be as free
>>> of platform dependencies and #ifdefs as much as possible. That is what
>>> HXThreadMessageSink and HXSocketSelector are for. (Ideally none of the
>>> other new socket api implementation will need updating/porting for
>>> non-Windows platforms, only these classes.)
>>> For Unix HXSocketSelector will need a function like
>>> "HXSockeSelector::Process()" akin to unix_net::process_idle(). This
>>> will need to be called in the thread message loop or in a timer
>>> function. Process() would probably be implemented very similarly to
>>> unix_net::process_idle(). Other code that is currently using
>>> HXSocketSelector (i.e., hxclientsockimp) should then "just work".
>>> HXThreadMessageSink basically maintains a list of callback handlers
>>> for thread messages. If you want to handle a certain thread message
>>> you look up the thread message sink and register a callback associated
>>> with the message. That is what the threaded socket code does. It
>>> registers to handle messages that the net thread posts back to the app
>>> thread. The resolver registers to receive messages that are posted
>>> back from the resolver thread(s). There is no dependency between the
>>> resolver, threaded socket and thread message dispatcher other than in
>>> a generic way in terms of participation in this thread message passing
>>> scheme. For example, if some other future code needs to pass messages
>>> between threads, it only needs to define a message and register a
>>> handler for the message with the thread message sink.
>>> Each thread message sink is associated with a unique handle. On
>>> windows the handle is actually an HWND. In fact, HXThreadMessageSink
>>> creates a dummy hwnd for hooking into the TLC-owned message loop. (On
>>> other platfroms the ID can most likely be 0.)
>>> When we pass a message to the main app thread (from net thread to
>>> client socket code running on app thread) we specify the handle of the
>>> thread message sink associated with the app thread in PostMessage().
>>> Because PostMessage() goes through the underlying WinAPI, the message
>>> ends up in the window associated with HXThreadMessageSink() and that
>>> is then forwarded to the handler for the message. The handler is
>>> registered at some point during initialization by the network code.
>>> In the case of calling WSAAsyncSelect() on Windows we lookup the
>>> thread message sink for the thread on which we want to receive select
>>> notifications and pass the HWND (the sink's ID) to WSAAsyncSelect.
>>> (First we register a callback to handle the select messages sent to
>>> the thread.)
>>> On non-windows platforms I was thinking that EventOccurred (or in the
>>> case of a non-main-app thread, the thread message loop that calls
>>> GetMessage()) would just forward messages through the thread message
>>> sink (which would call the appropriate handler for the message).
>>> Perhaps HXThread::DispatchMessage() on Unix (now not implemented)
>>> could just look up the message sink associated with its thread and
>>> pass the message on to the thread message sink to forward to the
>>> handler registered to handle the message. (HXThreadMessageSink could
>>> be hidden behind or part of HXThread even.)
>>> The TLS stuff is intended to ensure that if you call select on one
>>> thread (via HXSocketSelector) you get select notifications back on the
>>> same thread. It is also used to look up the HXThreadMessageSink
>>> per-thread instance associated with each thread. We can work around
>>> this by just keying on a thread id to associate thread-specific data.
>>> (In the old net implementation there was one select being done on one
>>> thread and these results were posted back to the main app thread. If
>>> we can assume select will only ever be needed/called on one thread
>>> then we do not need the TLS at all.)
>>> None of this is set in stone. As I said my primary goal was to
>>> encapsulate the stuff that differed significantly between platforms
>>> (thread message passing and select) and get things working on Windows,
>>> while at the same time trying to keep in mind eventual non-Windows
>>> platform needs.
>>>
>>>
>>>
>>> At 10:48 AM 5/24/2005, Nicholas Hart wrote:
>>>       
>>>> I was hoping that person would step up... :)
>>>>
>>>> Milko? Henry? Greg?
>>>>
>>>>
>>>> On Tue, 2005-05-24 at 06:23, John Stirling wrote:
>>>>         
>>>>> Any idea who the person to ask is ?
>>>>>
>>>>>
>>>>> On Mon, 2005-05-23 at 21:29, Nicholas Hart wrote:
>>>>>           
>>>>>> I think someone else on this list should have more info than I about
>>>>>> IPV6 plans for linux. I think there is more to do than just
>>>>>> implementing some functionality in the threading code (that part is
>>>>>> mostly just hooking up the helix APIs to the pthread APIs--not too
>>>>>> tricky).
>>>>>>
>>>>>>
>>>>>> On Mon, 2005-05-23 at 08:46, Philip Blundell wrote:
>>>>>>             
>>>>>>> That does seem to be the case, yeah; HELIX_CLIENT_IPV6_NET_API
>>>>>>>               
>>>> is only
>>>>         
>>>>>>> enabled #ifdef _WINDOWS. I tried switching this on for linux as
>>>>>>>               
>>>> well,
>>>>         
>>>>>>> but the resulting splay blows up at this code in
>>>>>>> hxthreadmessagesink.cpp:
>>>>>>>
>>>>>>> #if defined(_WINDOWS)
>>>>>>> // create instance for this thread
>>>>>>> hr = HXThreadMessageSink::Create(pStored);
>>>>>>> #else
>>>>>>> HX_ASSERT(false);
>>>>>>> hr = HXR_NOTIMPL;
>>>>>>> #endif
>>>>>>>
>>>>>>> which is called from
>>>>>>>               
>>>> CHXNetThreadClientNetServices::DoDriverInit. I
>>>>         
>>>>>>> didn't understand enough about the threading code to figure out
>>>>>>>               
>>>> what
>>>>         
>>>>>>> needs to be done to make all this work.
>>>>>>>
>>>>>>> p.
>>>>>>>
>>>>>>> On Thu, 2005-05-19 at 10:26 -0700, Nicholas Hart wrote:
>>>>>>>               
>>>>>>>> It's my understanding that IPV6 is not yet working on linux.
>>>>>>>>                 
>>>> There
>>>>         
>>>>>>>> was an overhaul of the networking layer for cayenne, which
>>>>>>>>                 
>>>> includes
>>>>         
>>>>>>>> IPV6, but due to resource limitations I believe it was only
>>>>>>>>                 
>>>> fully
>>>>         
>>>>>>>> implemented on Windows. I'm not up-to-date on our current
>>>>>>>>                 
>>>> plans, so
>>>>         
>>>>>>>> maybe someone else on this list can provide some more help.
>>>>>>>>
>>>>>>>>
>>>>>>>> On May 19, 2005, at 8:51 AM, John Stirling wrote:
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Is IPv6 expected to work on cayenne_1_5_0 (i586 or
>>>>>>>>>                   
>>>> arm-linux) ?
>>>>         
>>>>>>>>> I've just done a clean (i586) build using : build -m
>>>>>>>>> hxclient_1_5_0_cayenne_restricted -P
>>>>>>>>>                   
>>>> helix-client-all-defines -
>>>>         
>>>>>>>>> trelease
>>>>>>>>> -D 20050518 splay
>>>>>>>>> which seems to produce all the expected .so files
>>>>>>>>>
>>>>>>>>> We can play IPv4 streams ok with this build but can't get a
>>>>>>>>>                   
>>>> sample
>>>>         
>>>>>>>>> IPv6
>>>>>>>>> stream to work : http://virgin.6pack.org/vruk-lo-mp3
>>>>>>>>>
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Helix-client-dev mailing list
>>>>>>>>> [email protected]
>>>>>>>>>
>>>>>>>>>                   
>>>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>>>         
>>>>>>>> _______________________________________________
>>>>>>>> Helix-client-dev mailing list
>>>>>>>> [email protected]
>>>>>>>>
>>>>>>>>                 
>>>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>>> --
>>>> Nicholas Hart
>>>> [email protected]
>>>> Technical Lead, Helix Player
>>>> https://player.helixcommunity.org
>>>> http://www.real.com
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Helix-client-dev mailing list
>>>> [email protected]
>>>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>>>         
>>>       
>> _______________________________________________
>> Helix-client-dev mailing list
>> [email protected]
>> http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev
>>     
>
>

_______________________________________________
Helix-client-dev mailing list
[email protected]
http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev