Re: Piping, redirection and shellscipts: 3/5/2025 7pm Eastern Standard time

Steve Litt wrote on 2025-03-02 15:02:

> Yesterday I was tasked with getting every domain name I owned into
> my list at 444domains.com/domains , which is created by a
> shellscript and Python program that read a Yaml file and convert it
> to the web page. So I had to do the following:
> 
> * Screen scrape all my domain names from ionos, put in file
> 
> * Screen scrape all my domain names from Spaceship.Com, put in
> another file
> 
> * Use Vim to remove everything but the domains themselves
> 
> * Sort both files
> 
> * Create a Python program to sort my Yaml file
> 
> * Run diff between each file and the sorted domains of the Yaml file 
> - The output with the left arrows need to be added to Yaml file
> 
> * Re-sort the Yaml file
> 
> * Run the shellscript to create the web page.
> 
> This sounds straightforward

No, it does *not* sound straightforward!


Is it just me, or does it sound like a Rube Goldberg contraption in
software?



The list of all domain info should be in a DB or at least a text file,
for starters.


 From there, do the sort(s), build the YAML, etc. Why sort a YAML file?!?
Running `diff` and manually merging?


All this scraping seems pointless, why is the source of this data inside
web pages?