Re: [stack] Abstracting away the complexity is the goal.

chris glur <[email protected]> Mon, 22 Feb 2010 21:59:21 +0200
Newsgroups gmane.comp.lang.concatenative
Message-ID <[email protected]>
>How did you measure the complexity? I don't get it.
 The number of new concepts introduced.
...
> RSS allows you to easily parse out individual articles and comments.
>  That's its function.
 Perhaps I'll try it.
For decade/s I ignored Turbo-pascal & used my own P-code compiler, because
"turbo" was a fad-word. I've been thinking that blogs are just a passing fad
like sub-prime-loans.
...
> What sort of thing does the utility do? I'm guessing it runs a Google
> search qualified by "site:<URL>" for the 3 words...
 Yes, that an example of NOT complexity. You could see what it does.

> But why are you doing that?
1. use the power of google
2. reduce the search-space to <URL> only
3. further reduce to likely articles
   so that the size of text that I'll have to read is reduced.
...
]>>  Why is your pseudocode so wrong? That's horrific.
]> Yes switching syntax between: lisp, piped-data-flow, algol
]> is problematic.
> No, I mean all the FOR loops. You have a huge set of nested FOR loops.
> That's going to be _slow_.
IMO we mostly talk past each other.
If the algorithm demands nested FORs ...which this one does.
Except my text was wrong. Let's examine further ..
Just look at this mess:-
ContainsWord(
 ContainsWord(
  ContainsWord(
   AllFilesYoungerThanDays(15), <word1>), <word2> , <word3>)
which might STILL be wrong.

Each of the 4 functions returns a set-of-files/FileNames.

How much less complex [cognitive load] is a piped data-flow notation:

15 AllFilesYoungerThanDays |
<word1> ContainsWord |
<word2> ContainsWord |
<word3> ContainsWord |

  which can be directly translated from the natural language spec. of
"find which of new-files, contain word1 & word2 & word3".

So there is inevitable nesting:
  once the young-enough files have been found, they must be searched.
But the search space decreases by each step.

I'm less interested in the D.Knuth-like considerations of optimising the search
by eg. combining the search for the 3 [or n] words, rather than just doing
3 searches, that the HUMAN readability/maintainablility of the code.
...
> MIT's open classware is an incredible resource. It's not
> "old-timer", though; it's fairly recent.
Their tradition of clear writing [which I can't do] is old.
New technology just allowed open-classware; but the HUMAN capital behind it
adds the value. I was recently browsing my Samuelson [died recently at 94,
still giving lucid interviews till end] 1967 Edition. What brilliance !
...
> you wanted to use functions from libraries

YES! But they must be 'clean' as expounded in SICP.
IMO if they evolve naturally, they get over-run by entropy.
So they need to limited by formality.

Here's 2 example of very common bad:
* the matured and well-respected mail-client 'mutt' has a help-menu of
6 screens, ie. over 100 options; arranged ALPHABETICALLY [instead of
conceptually], in case someone wants to know "I wonder what mutt uses
the 'Z' key, or <esc> Q... for".

--> change topic <---
I mentioned that google is going to take over the world.
That's because search is the most important/needed function.
They've got the key to adding value to OUR explosive accumulatiuon of data.

Perhaps I have a special mental defficiency, but regular-expression-search
is the most fascinating/frustrating can-of-worms for me.
AGAIN I've spent hours, without success the find how to use sed to
<extract the string from the file: which starts with "start" and ends  with
"end", exclusively>.

Also ETH-oberon's RX search/replace utility kills me.

But related to 'sed' the following newly discovered cat-like [piping results
through transformers] method fascinates me.

Before I read this cat-mail-list I'd never have seen it this way:
I want to find the list of *sed* usenet groups,
 which is in the partition: 'something about "11" ';
 in the file: 'something about "*ot*st" ' eg. TotalLst, totalList...etc;
 in the dir: 'something about "*net*" '.

Here's the data-flow/piping sequence:
df <-- gives me the list of mounted partitions
 | grep 11 <--- gives me the line mentioning the one I want
 <missing> <--- the code that I can't do yet to extract a substring via sed
 find <the 3 stage pipe above> -name "*ot*st" <--something about "*ot*st"
 | grep "*net*" <--- something about "*net*"

----------
So the 5 steps described in the 2 above paragraphs even with messy unix
utilities would give < 100 chars of code.
The beauty/danger-of-BASIC-like-illusion is that you can manually test it
 easily by using the 'history key' to pop-up previous commands and modify
 or wrap them. Ie. a cutNpaste effect, encouraging a dumbed-down, suck-it-&-see
 approach, which I think is the reason for all the sed-tutors who can ride a
 bicycle but don't KNOW how to ride a bicycle. Ability from practicing examples
 is NOT a substitute for knowledge of the underlying model.

So now that the 5 simple steps have found the file, it remains only to:
 <list all lines [since there's one newgroupId per line] containing *sed*>
 ie. `| grep "*sed*"

So the data-flow looks like this"
 partitionList -> with "11" -> trim-out -> with "*ot*st" -> with "*net*"
    ->  containing "*sed*"

Pity I couldn't fit it on 1 line-len < 80.
And the reason why I prefer it to the algol style programming that I'm used
to, is that it's much simpler, and therefore more testable and maintainable.

So, the zillions of year-lines of confirmed unix code , prove the viability
of the concept.  Except the unix utilities just evolved chaotically and
don't fit together cleanly, as a system which was designed from the ground,
would.
==================
Yes it's flat which is the basis of [reason for] the simplicity.
The nesting comes in the called functions,
 which presumably can nestedly call other routines.

I'm not convinced that adding control stuctures to this data-flow-view,
is viable, because it would destroy the one-dimensional simplicity.

IMO forth is a nightmare to read.

Whereas I'm proposing to use flat data-flow to call functions,
as you know, unix uses block-structured algol-style scripts, with
data-flow-tricks to get get values, like [conceptually]
 append the date-field of the line mentioning "dog" to the
  weight-field of the line mentioning "cat".

== Chris Glur.