[JIRA] Updated: (CC-991) Piping STDOUT->STDIN of scripts (piped exec builder)

     [ http://jira.public.thoughtworks.org/browse/CC-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Tihelka updated CC-991:
------------------------------

    Attachment: PipedExecBuilder_13_WinFix.patch

Thank you, Dan, it seems OK (at least, the tests are passed :-))

Now, I'm attaching the last patch (PipedExecBuilder_13_WinFix) which fixes the issue with PipedExecBuilder failing tests on windows. I have tried it several times with various configurations, so I hope that this last issue is finally resolved.

In addition, I have slightly generalized StreamPumper class - now it can pump either binary or text  streams, or both at the same time (there are also new tests for it). It is used in PipedExecBuilder, and it replaces quite ineffective way, where STDOUT data were passed through StdoutBuffer (which would compress then and decompress them immediately in case of GzippedStdoutBuffer). Also, it allowed to remove StdioPumper class from ScriptRunner.

Also, as minor changes, I have lowered the number of sentences generated in PipedExecBuilder test, since on my Windows the tests run significantly slower than on Linux (I don't know why ...).


> Piping STDOUT->STDIN of scripts (piped exec builder)
> ----------------------------------------------------
>
>                 Key: CC-991
>                 URL: http://jira.public.thoughtworks.org/browse/CC-991
>             Project: CruiseControl
>          Issue Type: New Feature
>          Components: Core Application
>         Environment: Any
>            Reporter: Daniel Tihelka
>         Assigned To: Dan Rollo
>            Priority: Major
>         Attachments: Documentation.patch01, Documentation_02.patch, PipedExecBuilder-07_1.patch, PipedExecBuilder.java, PipedExecBuilder.patch01, PipedExecBuilder.patch02, PipedExecBuilder.patch03, PipedExecBuilder.patch04, PipedExecBuilder.patch05, PipedExecBuilder.patch06, PipedExecBuilder.patch07, PipedExecBuilder_08.patch, PipedExecBuilder_09.patch, PipedExecBuilder_10.patch, PipedExecBuilder_11.patch, PipedExecBuilder_12.patch, PipedExecBuilder_13_WinFix.patch, PipedExecBuilderTest.java, StdoutBuffer.java, StdoutBufferTest.java
>
>   Original Estimate: 2 weeks
>  Remaining Estimate: 2 weeks
>
> The build process may sometime consist of the set of scripts piped one into another (the unix-way of filtering). This issue targets CruiseControl extension which would allow it.
> Almost a year ago, I have created PipedExecBuilder class as 3rd party CC plugin. It contains several scripts to execute, while the output of any script can be piped into the input of an arbitrary number of other scripts. Moreover, the execution of a script may be hold until another script is running (to prevent several scripts with large memory requirements running at the same time).
> The configuration may look as follows:
> <schedule>
>    <pipedexec>
>      <exec id="" command="" args="" />
>      <exec id="" command="" args="" pipefrom="" />
>      <exec id="" command="" args="" waitfor="" />
>      <exec id="" command="" args="" pipefrom="" waitfor="" />
>    </pipedexec>
> </schedule>
> where the <exec /> tag is the same as the original <exec /> tag, with additional attributes:
> - id - unique identifier of the script for the piping
> - pipefrom - the id of script from which to pipe stdout into the stdin of the current script (optional)
> - waitfor - the id of script to wait for - it holds the execution of the current script until the script which we wait for terminates.
> The pipedexec builder tries to run as many scripts in parallel as possible, while it finishes when all the scripts were finished, or when a script failed.
> For further details see cruisecontrol-devel mailing list thread:
> http://sourceforge.net/mailarchive/forum.php?thread_name=201005271341.32553.dtihelka%40kky.zcu.cz&forum_name=cruisecontrol-devel
> Kind regards,
> DanT

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

------------------------------------------------------------------------------
Magic Quadrant for Content-Aware Data Loss Prevention
Research study explores the data loss prevention market. Includes in-depth
analysis on the changes within the DLP market, and the criteria used to
evaluate the strengths and weaknesses of these DLP solutions.
http://www.accelacomm.com/jaw/sfnl/114/51385063/