RE: Mini-SOD for unifying producer and client logging

Steve,

Replies inline below...

=======================================
Eric Hyche ([email protected])
Principal Engineer
RealNetworks, Inc.


>-----Original Message-----
>From: Steve McMillen [mailto:[email protected]]
>Sent: Wednesday, March 04, 2009 9:07 AM
>To: [email protected]
>Cc: [email protected]; [email protected]
>Subject: Re: Mini-SOD for unifying producer and client logging
>
>I did not see the Producer ActiveX control mentioned in the SOD.  If the cmd line app has to be
>modified, doesn't the ActiveX also?
>

Yes, any producer application would need to be modified, including
the ActiveX. I'm going ahead and making the changes for the command-line
producer, since that's what we need immediately.

The changes are pretty minor. It's essentially just doing log initialization
and shutdown in a different place (when called by the media platform).

>I notice the log translation file was mentioned but just to make sure, can all messages still come
>from a resource file and just the message ID is stored in the code?
>

Yep, the current log messages in the translation file don't have
to be modified. All I did was add the client-related functional
areas definitions so that client messages could be translated
as well.

>Following is the Spec on Producer logging: Logging
><http://tools/Groups/Tools/Projects/RealProducer/Phoenix/Specs/Logging> . I can't quite tell from the
>SOD if the implementation is resulting in fundamental changes with the producer logging but just to
>make sure, will the following still work:
>- Log filtering by the SDK - a client can register to receive only a certain set of messages (this is
>desirable if we implement remote encoder clients)

Yes - that still works.

>- Logging for multiple jobs - Several instances of producer can run and log to the same file. The job
>filename for each instance is recorded as a field in the log file (this is critical for our app needs)
>

Yep - that still works.

>From a Producer SDK app's point of view the *only* thing that changes
is when the log system is initialized and shutdown. Everything
stays the same. (Well, getting the log system and the file observer
is now easier -  you don't have to do the messy DLLAccess stuff
any more).

Eric

>
>-- Steve
>
>on 3/2/09 8:19 AM Eric Hyche wrote the following:
>
>	All:
>
>	Please find attached a mini-SOD for the necessary
>	changes for:
>
>	a) getting logging working on Producer builds; and
>	b) allowing Helix client plugins log messages simultaneously with
>	   producer sdk log messages.
>
>	I have already discussed these changes with Sujeet and Milko.
>	Sujeet: this is only a slight modification from what we
>	discussed last week.
>
>	I started on these changes last week and should
>	be finished by EOD Tuesday. If you have any questions,
>	comments, or feedback, please let me know.
>
>	Eric
>
>	=======================================
>	Eric Hyche ([email protected])
>	Principal Engineer
>	RealNetworks, Inc.
>
>
>
>
>________________________________
>
>
>	Statement of Design for Unifying Producer and Client Logging
>	-------------------------------------------------------------------
>	The Producer SDK logging system was adopted by the Helix client
>	several years ago. At the time, we kept several things about
>	the implementation separate - different macros, different ways of
>	initializing, different way of loading the logging system DLL, etc.
>	We kept this separation because so that both systems could continue
>	to operate with no changes.
>
>	Now that the Producer SDK is integrated with the Helix client Atlas
>	platform, we need to unify how both systems use logging.
>
>	Here is an overview of the current way that the Helix
>	client uses logging (before the changes I'm proposing in this
>	document).
>
>	- Logging observer plugins implement IHXPlugin and IHXPluginProperties
>	  and in IHXPluginProperties::GetProperties(), they advertise a
>	  property named "LoadAtStartup" which has a value of "MediaPlatform".
>	- When the media platform is initialized via IHXMediaPlatform::Init(),
>	  is queries the plugin handler for all plugins with a property
>	  named "LoadAtStartup" equal to "MediaPlatform". It then takes
>	  the plugin enumerator returned by the plugin handler and loads
>	  each of these plugins and calls IHXPlugin::InitPlugin() on them.
>	- Right now there is only one observer plugin - the observer which
>	  writes log statements to a file in common/log/logobserverfile.
>	  When this plugin gets a call to IHXPlugin::InitPlugin(), it
>	  reads the following preferences rooted at "Logging/File"
>	  in the preference tree:
>	    "Enabled"        - enable/disable this observer
>	    "DeliveryThread" - use a separate thread for delivery of log messages
>	                       to the observers
>	    "Filename"       - filename of the log file
>	    "LevelFilter"    - only display logging messages of this level
>	    "AreaFilter"     - comma-separated list of logging 4cc's (listed
>	                       in common/include/ihxtlogsystem.h)
>	    "Separator"      - character used to separate fields in log statements
>	  These values have defaults if there is no preference present.
>	- Using these values, the appropriate calls to IHXTFileObserver are
>	  made like IHXTFileObsever::Enable(), IHXTFileObserver::SetCategoryFilter(),
>	  IHXTFileObserver::SetSeparator(), and finally IHXTFileObserver::Init()
>	  is called, passing in the filename of the log file.
>	- Inside IHXTFileObserver::Init(), the file observer uses DLLAccess to
>	  load the logging system dll (log.dll in common/log/logsystem). It then
>	  calls It then calls the "RMACreateLogSystem" entrypoint to get a pointer
>	  to the IHXTLogSystem interface. From the IHXTLogSystem pointer, it
>	  calls IHXTLogSystem::GetObserverManagerInterface() to get the observer
>	  manager than then uses the observer manager to subscribe the file
>	  observer into the log system.
>	- In code that wants to make logging statements, somewhere in the binary
>	  there must be a call to HX_ENABLE_LOGGING(pContext). The code in hxtlogutil
>	  library (common/log/logutil) will take that context, QI it for IHXCommonClassFactory,
>	  and then call CreateInstance(CLSID_IHXDllAccess,), and then use that
>	  IHXDllAccess object to load the logging dll (log.dll) and calls the
>	  "RMACreateLogSystem" entrypoint to get an IHXTLogSystem pointer. It then
>	  calls IHXTLogSystem::GetWriterInterface() to get the writer interface
>	  which it sets into a global static variable.
>	- So the Helix client's usage of the logging system is "automatic". As long
>	  as logging statements are present, then if log.dll and logobserver.dll
>	  are present in the plugins directory, then the logging file appears.
>
>	The Producer SDK takes a much more "manual" approach to using logging.
>
>	- The application manually loads the logging system dll (log.dll), gets
>	  the "RMACreateLogSystem" entrypoint which creates a singleton global
>	  variable IHXTLogSystem pointer inside the log.dll binary (this is the
>	  same as in the client). It also then manually loads the file observer
>	  by calling CreateInstance(CLSID_IHXTFileObserver,) on its factory.
>	- From the UI or command line parameters, it sets the values on
>	  IHXTFileObserver and then calls IHXTFileObserver::Init() which subscribes
>	  the file observer to the log system same as in the client. However,
>	  it calls the "RMAGetLogSystem" entrypoint which returns a IHXTLogSystem
>	  pointer ONLY if one has already been created. In other words, it assumes
>	  that the log system has already been created by the application.
>	- Code that wants to write logging statements use different macros
>	  to write log statements, although these macros eventually map to the
>	  same underlying function call of IHXTLogWriter::LogMessage().
>	- Also, in code that wants to write logging statements, the Producer SDK
>	  plugins do not require a context or an HX_ENABLE_LOGGING macro.
>	  That's because they maintain a global static DLLAccess class which
>	  is used to load the logging dll.
>
>	Previously we used the presence of the HELIX_FEATURE_LOG_STATICDLLACCESS
>	to distinguish between these two ways of using the log system.
>	The HELIX_FEATURE_LOG_STATICDLLACCESS define was present in helix-producer-all-defines
>	but not present in helix-client-all-defines. However, now we are
>	building with the same profile, so we would like to be able to
>	have both Helix client logging messages and Producer SDK logging
>	messages in the same log file.
>
>	Therefore, I propose we unify these two approaches such that both can
>	continue to operate as they desire but we don't have this compile-time
>	decision of whether to support either Helix client logging
>	or Producer SDK logging. Here are the changes I propose:
>
>	1) Make the logging DLL an IHXComponentPlugin for Atlas. That means that
>	   the media platform can create an instance of IHXTLogSystem simply
>	   by calling its CreateInstance() method. The previous "RMACreateLogSystem"
>	   and "RMAGetLogSystem" will still be exposed, but they will return
>	   HXR_NOTIMPL and will throw an HX_ASSERT() if called.
>
>	2) Change the log observers (currently only the file observer)
>	   IHXPluginProperties to advertise a "PluginType" of "LogObserver"
>	   instead of "LoadAtStartup" = "MediaPlatform".
>
>	3) We define a new interface called IHXLogSystemManager. The methods
>	   of IHXLogSystemManager are:
>
>	   InitializeLogSystem() - this loads the log system by calling
>	       CreateInstance(CLSID_IHXTLogSystem,) on the media platform
>	       IHXCommonClassFactory.
>	   TerminateLogSystem() - this calls IHXTLogSystem::Shutdown()
>	       and then releases the log system.
>	   GetLogSystem(REF(IHXTLogSystem*) rpLogSystem) - this returns the log system
>	   InitializeLogObservers() - Load, configure, and subscribe
>	       all log observers
>	   TerminateLogObservers() - Unsubscribe and unload the log observers
>
>	4) We add another "basic service" to the Atlas platform - log
>	   system management. What this means is:
>
>	    a) In IHXMediaPlatform::Init(), the media platform checks
>	       to see if the context that the application passed in
>	       (if present) supports IHXLogSystemManager.
>	         - If it does, then it uses that IHXLogSystemManager
>	           interface and holds a reference to it in the media platform.
>	         - If it does not (or the context is NULL), then the media
>	           platform creates its own internal implementation
>	           of IHXLogSystemManager.
>	       In either case, the media platform can now be QI'd for
>	       IHXLogSystemManager.
>
>	    b) In IHXMediaPlatform::Init() after we obtain an IHXLogSystemManager
>	       (either from the Init() context or by creating our own internal
>	       instance), then the media platform calls
>	       IHXLogSystemManager::InitializeLogSystem() and then
>	       IHXLogSystemManager::InitializeLogObservers().
>
>	    c) In IHXMediaPlatform::Close(), the media platform calls
>	       IHXLogSystemManager::TerminateLogObservers() and then
>	       IHXLogSystemManager::TerminateLogSystem().
>
>	5) The Helix client will have a default implementation of
>	   IHXLogSystemManager located in common/log/logutil. The default
>	   implementation will do what the client does now - configure the
>	   file observer from preferences.
>
>	6) Producer applications will need to be modified to implement
>	   IHXLogSystemManager. The Producer could re-use some parts
>	   of the client's IHXLogSystemManager implementation. However,
>	   it's implementation would set the file observer properties
>	   from user input instead of reading from preference. Also, the
>	   command-line producer would create and initialize the
>	   "screen logger" as it currently does.
>
>	7) common/log/logutil/hxtlogutil.cpp would be changed so that
>	   when we call HXEnableLogging(pContext), we just QI pContext
>	   for IHXLogSystemManager and then call
>	   IHXLogSystemManager::GetLogSystem() to get an IHXTLogSystem
>	   pointer. Everything else (getting the IHXTLogWriter and
>	   IHXTLogSystemContext interfaces from IHXTLogSystem) would
>	   stay the same.
>
>	8) We will update the logmessages.xml file:
>	     a) to use 4cc's for the "id" attribute in <FunctionalArea>
>	        elements instead of ordinal numbers;
>	     b) to add <FunctionalArea> elements for the client logging 4cc's
>
>	9) Update LoadTranslationFile() method in
>	   common/log/logsystem/hxttranslationcentre.cpp to parse the
>	   new "id" attribute (which are now 4cc's).
>
>
>	Tasks:
>
>	1) Make log.dll into an Atlas component plugin (3 hours, DONE)
>	2) Change logobserver plugin properties (0.1 hour)
>	3) Change media platform to use IHXLogSystemManager (1 hour)
>	4) Provide default implementation of IHXLogSystemManager (2 hours)
>	5) Modify command-line producer to implement IHXLogSystemManager (2 hours)
>	6) Update logmessages.xml file (1 hour)
>	7) Update LoadTranslationFile() to handle new "id" attribute (1 hour)
>