Data Structure for use vs storage

1) Instead of SQLITE you can also try B-TREES in Lisp (SQLITE is basically B-TREES).
A zero dependency one file solution https://github.com/ebobby/cl-bplustree:

(defparameter *my-tree* (bplustree-new 4 :key (lambda (r) (parse-integer r))))

(bplustree-insert-many          *my-tree*  "1337" "212" "32" "311" "52")  ;;  #<B+ TREE: 2 {61BDAB45}>
(bplustree-search 311           *my-tree*)                                ;; "311"
(bplustree-search-range 100 400 *my-tree*)                                ;;  ("212" "311")

;;----------------------------------------------------------------------------------------

2) Another approach by the same author: https://github.com/ebobby/cl-simple-table

"Simple table data structure and functions to work with them.
Specially useful to play around with tabular data files (like csv or tsv)."

---

It supports key-based access, no need to remember column indices.

(simple-table:read-tsv #P"example.tsv")
#(#("Year" "Make" "Model") #("1997" "Ford" "E350") #("2000" "Mercury" "Cougar"))


Worth trying, especially if you plan to delegate parts of the work to LLMs later.

A practical workflow:

a) append four cl-simple-table files into a single small file.

b) remove package declarations such as (in-package :cl-simple-table).
Call functions directly, e.g. (read-tsv #P"example.tsv");  without 'simple-table:'

d) attach this cleaned file together with your own functions to all conversations.
By doing so, LLMs are not guessing, they can analyze the full context of your project.


Kind Regards,
Adrian Pasieka