Re: unboxed structure fields (or class slots)

The following exercise may help Martin decide if and how to implement LLMs.

There is an ARC-AGI challenge, and the simplest task seems to be b7999b51.
https://anokas.substack.com/p/o3-and-arc-agi-the-unsolved-tasks


This is my attempt to evaluate LLMs' 'reasoning'.


1) LLM-s can't synthesize shapes from raw JSON data, so they are given a lists of shapes.

The first IN-OUT.
 (DEFPARAMETER input-1-facts '(((2 2 C) (2 3 C) (2 4 C) (3 2 C) (3 4 C) (4 2 C) (4 4 C) (5 2 C) (5 4 C) (6 2 C) (6 4 C) )
                               ((3 3 O) (4 3 O) (5 3 O) (6 3 O) (7 3 O) (8 3 O) (9 3 O) (10 3 O) (11 3 O) (12 3 O) (13 3 O) (14 3 O))

                               ((4 6 F) (4 7 F) (4 8 F) (4 9 F) (4 10 F) (5 6 F) (5 7 F) (5 8 F) (5 9 F) (5 10 F) (6 6 F) (6 7 F) (7 6 F) (7 7 F)
                               (8 6 F) (8 7 F) (9 6 F) (9 7 F) (10 6 F) (10 7 F))
                               ((6 8 N) (6 9 N) (6 10 N) (6 11 N) (7 8 N) (7 9 N) (7 10 N) (7 11 N) (8 8 N) (8 9 N) (8 10 N) (8 11 N) (9 8 N) (9 9 N) (9 10 N)
                               (9 11 N) (10 8 N) (10 9 N) (10 10 N) (10 11 N) (11 8 N) (11 9 N) (11
                               10 N) (11 11 N) (12 8 N) (12 9 N) (12 10 N) (12 11 N) (13 8 N)
                               (13 9 N) (13 10 N) (13 11 N))))

 (DEFPARAMETER output-1-facts '(((0 0 F) (0 1 F) (0 2 F) (0 3 F) (0 4 F))
                                ((1 0 N) (1 1 N) (1 2 N) (1 3 N))
                                ((2 0 C) (2 1 C) (2 2 C))
                                 ((3 0 O))))


2) Gemini generates the function which maps input-1-facts into input-2-facts correctly (with human help).

 (DEFUN arc-transformer-1 (input-facts)
  (LET* ((colors '(C O F N))
         (objects (MAPCAR (LAMBDA (col) (REMOVE-IF-NOT (LAMBDA (p) (EQ (THIRD p) col)) input-facts))
                                     colors))

         (object-data (MAPCAR (LAMBDA (obj) (LET ((cols (MAPCAR #'SECOND obj)))
                                              (LIST :color (THIRD (CAR obj))
                                                    :width (1+ (- (REDUCE #'MAX cols) (REDUCE #'MIN cols))))))
                               objects))

         (sorted-objects (STABLE-SORT (COPY-LIST object-data)  #'> :KEY (LAMBDA (x) (GETF x :width))))

         (final-output (LOOP FOR obj IN sorted-objects FOR row-idx FROM 0
                          COLLECT (LOOP FOR col-idx FROM 0 TO (1- (GETF obj :width))
                          COLLECT (LIST row-idx col-idx (GETF obj :color)))) ) )

             final-output))

 (DEFUN flatten-level (level list-1)
  "Flatten possibly nested LIST a given number of LEVELs (or to the end)."
 (LABELS ((rec (x  acc depth)
   (COND ((NULL x) acc)
         ((ATOM x) (CONS x acc))
         ((AND depth (ZEROP depth)) (APPEND x acc))
         ((rec (CAR x) (rec (CDR x) acc depth) (WHEN depth (1- depth)))) )  ))
          (rec list-1 NIL level) ))


RESULT:  (arc-transformer-1 (flatten-level 1 input-1-facts)) ;; correct

         (((0 0 F) (0 1 F) (0 2 F) (0 3 F) (0 4 F))
          ((1 0 N) (1 1 N) (1 2 N) (1 3 N))
          ((2 0 C) (2 1 C) (2 2 C))
          ((3 0 O)))


3) Logically, this successful template should be reused for the next steps.
   But Gemini ignores it and generates a brand-new function for the second IN-OUT task.

(DEFUN arc-transformer-2 (input-facts)
  (LET* ((flat-facts (REDUCE #`APPEND (REDUCE #`APPEND input-facts)))
         (all-colors (REMOVE-DUPLICATES (MAPCAR #`THIRD flat-facts)))
         (color-counts (MAPCAR (LAMBDA (col)  (LIST :color col  :count (COUNT col flat-facts :KEY #`THIRD)))  all-colors))
         (sorted-colors (SORT (COPY-LIST color-counts) #`> :KEY (LAMBDA (x) (GETF x :count))))
         (output-map `((H . 4) (F . 3) (S . 1)))
         (final-cells `()))
    (LET ((current-row 0)
          (current-col 0)
          (final-output (LOOP FOR color-entry IN sorted-colors
            FOR color = (GETF color-entry :color)
            FOR target-count = (CDR (ASSOC color output-map))
            DO (LOOP REPEAT target-count
                     DO (PUSH (LIST current-row current-col color) final-cells)
                        (INCF current-col)
                        (WHEN (>= current-col 4)
                          (SETF current-col 0)
                          (INCF current-row)))))

      (LIST (LIST (REVERSE final-cells)))))

This arc-transformer-2 function has a different, incorrect structure and fails entirely.


SUMMARY:
1) Illusion of Continuity: Prompts create a fake sense of "ongoing thought."
2) No Reflective Thinking: LLMs do not review or validate their previous logic.
3) Data Over Logic: LLMs restart every task using training data, not established facts.

There are attempts to bridge this gap via Recursive Reasoning Models and other concepts.


I’m calling 2026 'the year of the CLI and REPL', defined by the history and beauty of Lisp.
___________________________________________________________________________________________


  https://code.claude.com/docs/en/cli-reference
  Command             Description 
  claude              Start interactive REPL 
  claude "query"      Start REPL with initial prompt.
  



Kind Regards,
Adrian W. Pasieka