Re: Creating custom propertis plugin

Hi Dan T,

Sorry for the slooow reply. Mine is not the most informed opinion either...

I think the lack of support for a "pluggable" properties handler is not 
a design intent, but just a side effect of nobody needing one before now.

That said, there was some talk a while back about eventually supporting 
pluggable build queues too (allowing us to swap in a queue 
implementation as desired).

So I don't know of any reason not to extend the api to support new a 
PropertiesPlugin. Since this is so core to the system, it should be well 
unit tested.

Also keep in mind such a plugin needs to play nice with the existing 
property resolution/inheritance system (or at least clearly document 
where behaviors differ, if at all). Probably not an issue, but worth 
verifying.

Dan (R)


Date: Wed, 27 Jul 2011 09:14:55 +0200
From: Daniel Tihelka <[email protected]>
Subject: [Cruisecontrol-devel] Creating custom propertis plugin
To: [email protected]

Dear developers,

I would like to create properties reader which will be able to parse 
general
files and extract properties from them based on regular expressions. The 
aim is
to make our build projects more robust, since they use many various (mostly
python and C++) command line utilities with options set like:

--keep-feats 'unit:dur:IPS:score'

Of course, the individual strings are defined as constants to avoid their
repeating in the source files of the utilities. For example:

#define KEY_UNIT "unit"
...

if (item[i] == KEY_UNIT) {
     ...
}


So, it would be nice to have a set of properties in CC holding the values.
Currently, we define such properties "manually" in the CC project, 
facing build
failures everywhere someone decides to change the name. However, the 
property
values could fairly easily be parsed automatically from the appropriate 
file(s)
(and the automatic properties update framework
http://jira.public.thoughtworks.org/browse/CC-952, which I helped to made
before, would keep it up-to-date).

Well, to build such a general properties reader, I have created a class:

class RegexProperties implements PropertiesPlugin, ResolverUser
{
   ....
}

and implemented it. To test it, I have created a test project like:

<cruisecontrol>
   <plugin name='regexproperties' classname='zcu.kky.RegexProperties' />
   <project name="regex_properties" buildafterfailed="true">
     <regexproperties file="regexproperties.txt"  pattern="\s*([^\s]+)\s*::
\s*([^\s+])" name="try.\1" value="\2"/>
     <schedule interval="600" >
       <exec  command="echo"  args="WE HAVE: try.prop1 = ${try.prop1}"/>
     </schedule>
   </project>
</cruisecontrol>

and the regexproperties.txt file like:

prop1   :: WORKING!


But, project loading fails with the following message:

net.sourceforge.cruisecontrol.CruiseControlException: error configuring 
project
pokus_regex_properties
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.handleProject(CruiseControlConfig.java:422)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.parse(CruiseControlConfig.java:171)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.<init>(CruiseControlConfig.java:187)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.add(CruiseControlConfig.java:303)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.handleIncludedProjects(CruiseControlConfig.java:202)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.parse(CruiseControlConfig.java:160)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.<init>(CruiseControlConfig.java:187)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.add(CruiseControlConfig.java:303)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.handleIncludedProjects(CruiseControlConfig.java:202)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.parse(CruiseControlConfig.java:160)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.<init>(CruiseControlConfig.java:132)
         at
net.sourceforge.cruisecontrol.config.XMLConfigManager.loadConfig(XMLConfigManager.java:86)
         at
net.sourceforge.cruisecontrol.config.XMLConfigManager.<init>(XMLConfigManager.java:78)
         at
net.sourceforge.cruisecontrol.CruiseControlController.setConfigFile(CruiseControlController.java:93)
         at 
net.sourceforge.cruisecontrol.Main.createController(Main.java:146)
         at net.sourceforge.cruisecontrol.Main.start(Main.java:113)
         at
net.sourceforge.cruisecontrol.launch.Launcher.run(Launcher.java:259)
         at
net.sourceforge.cruisecontrol.launch.Launcher.main(Launcher.java:117)
Caused by: net.sourceforge.cruisecontrol.CruiseControlException: Nested
element: 'regexproperties' is not supported for the <project> tag.
         at
net.sourceforge.cruisecontrol.PluginXMLHelper.configureObject(PluginXMLHelper.java:215)
         at
net.sourceforge.cruisecontrol.PluginXMLHelper.configure(PluginXMLHelper.java:134)
         at
net.sourceforge.cruisecontrol.PluginXMLHelper.configure(PluginXMLHelper.java:92)
         at
net.sourceforge.cruisecontrol.ProjectXMLHelper.configurePlugin(ProjectXMLHelper.java:95)
         at
net.sourceforge.cruisecontrol.CruiseControlConfig.handleProject(CruiseControlConfig.java:420)
         ... 17 more


Examining it more deeply, I have found that the problem is in
PluginXMLHelper#configureObject(Element, Object, boolean), around lines
200-220, where there is no "added" found. I have look at what "adders" are,
and realized that they are "add*" methods from ProjectConfig, like
add(LabelIncrementer), add(Listeners), but there is no valid
add(PropertiesPlugin) or similar method.


So, the question is - is it intended behavior (i.e. properties cannot have
form of a plugin) or a kind of bug? Where and how to fix it? If possible,
please, give me some suggestions and will try to implement it.

I believe that it could be handy to have a possibility to create custom
properties plugins. I can, for example, image the situation, when 
properties
are stored in a database.

Thank you for your help.
Best regards,
Dan T.


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1