| Newsgroups |
gmane.editors.sed.user |
| Message-ID |
<[email protected]> |
So another variation! puresed, I like the name, is faster. However, I
would not call it "remarkable". 2:1 (what I see, based on multiple
runs), or even 4:1 (close to what you saw) is often not, IMO, because
the absolute difference is so small. I would suggest speed is often
overrated as a criterion. In this case, it seems not important:
$ wc big.txt
100005 300015 6200301 big.txt
$ time process.sh big.txt 3
real 0m0.416s
user 0m0.392s
sys 0m0.008s
$ time puresed.sh big.txt 3
real 0m0.293s
user 0m0.396s
sys 0m0.020s
IMO, the pedestrian cut-paste solution is better, not because it was my
suggestion, I'm here to learn and share, but because it works, it's
obvious what happens at each step, and it's fast. I wish it was shorter,
but we can't have everything.
Daniel
On 12/12/2015 12:53 PM, Thierry Blanc [email protected] [sed-users]
wrote:
> this is a parameter driven pure sed script (apart from the oxymoron ;-) ).
> The speed is remarkable. See below
>
> puresed.sh
> #!/bin/bash
>
> COL=$2
> ((COL--))
>
> MAXCOL=$(sed -n 's|,|\n|gp;q' $1 |wc -l)
> ((MAXCOL--))
>
> let "COL2 = $MAXCOL - $COL"
>
> n1="$(printf "%${COL}s" |sed 's| |n;|g')"
> n2="$(printf "%${COL2}s" |sed 's| |n;|g')"
> N="$(printf "%${MAXCOL}s" |sed 's| |N;|g')"
>
> sed 's|,|\n|g' $1 |sed -r "$n1;s|[A-Z]+|\L\u\0|g;$n2" |\
> sed "$N;s|\n|,|g" > output
>
> ---
>
> process.sh
> #!/bin/sh
>
> col_to_sed=3
>
> prev_col=`expr $col_to_sed - 1`
> next_col=`expr $col_to_sed + 1`
>
> cut -d \; -f -$prev_col < $1 > cols_before.txt
> cut -d \; -f $col_to_sed < $1 > cols_.txt
> cut -d \; -f $next_col- < $1 > cols_after.txt
>
> sed -e '1b;s/.*/\L&/;s/\<./\u&/g' < cols_.txt > cols_new.txt
>
> paste -d\; cols_before.txt cols_new.txt cols_after.txt > output.csv
> ---
>
>
> $ ./puresed.sh test.csv 3
> .354332867
> $ ./puresed.sh test.csv 3
> .347485474
> $ ./puresed.sh test.csv 3
> .345228981
> $ ./puresed.sh test.csv 3
> .398209122
> $ ./process.sh test.csv 3
> 1.250124104
> $ ./process.sh test.csv 3
> 1.242909844
> $ ./process.sh test.csv 3
> 1.236377148
> $ ./process.sh test.csv 3
> 1.240610077
>
> On 12/12/15 19:19, Daniel Goldman [email protected] [sed-users] wrote:
>> Thanks for doing the speed test. Of course, another disadvantage of the
>> while-loop script is that it hard codes (a b c d). To my understanding,
>> the cut-paste script can be totally parameter-driven.
>>
>> Just double-checking: The post shows IFS=',' used for while-loop script.
>> Was intended IFS=';' used in actual speed test?
>>
>> Daniel
>>
>> On 12/12/2015 12:57 AM, Thierry Blanc [sed-users] wrote:
>>> speed test
>>>
>>> My concerns about a shell while loop were correct. The while-do loop is
>>> much slower.
>>>
>>> 1.079455912 vs. .221387482
>>>
>>>
>>>
>>> while-loop script:
>>>
>>> #!/bin/sh
>>>
>>> START=$(date +%s.%N)
>>> while IFS=',' read a b c d
>>> do
>>> c="$(echo "$c" | sed -r 's/(.*)/\L\1/; s/\<(.)/\u\1/g' )"
>>> echo "$a ; $b ;$c ; $d"
>>>
>>> done < $1 > $2
>>>
>>> END=$(date +%s.%N)
>>> echo "$END - $START" | bc
>>>
>>> #endofscript
>>>
>>> cut-paste script:
>>>
>>> #!/bin/sh
>>>
>>> START=$(date +%s.%N)
>>>
>>> col_to_sed=3
>>>
>>> prev_col=`expr $col_to_sed - 1`
>>> next_col=`expr $col_to_sed + 1`
>>>
>>> cut -d \; -f -$prev_col < $1 > cols_before.txt
>>> cut -d \; -f $col_to_sed < $1 > cols_.txt
>>> cut -d \; -f $next_col- < $1 > cols_after.txt
>>>
>>> sed -e '1b;s/.*/\L&/;s/\<./\u&/g' < cols_.txt > cols_new.txt
>>>
>>> paste -d\; cols_before.txt cols_new.txt cols_after.txt > output.csv
>>>
>>> END=$(date +%s.%N)
>>> echo "$END - $START" | bc
>>>
>>> #endofscript
>>>
>>>
>>
>> ------------------------------------
>> Posted by: Daniel Goldman <[email protected]>
>> ------------------------------------
>>
>
>
>
> ------------------------------------
>
> ------------------------------------
>
------------------------------------
------------------------------------
--
------------------------------------
Yahoo Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/sed-users/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/sed-users/join
(Yahoo! ID required)
<*> To change settings via email:
[email protected]
[email protected]
<*> To unsubscribe from this group, send an email to:
[email protected]
<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/