CC Mode 5.35.1; cc-mode slow performance

Hi

cc-mode can be slow (appear to hang) on many of our files. The pattern that
slows down emacs seems to be related to "lists" like code such as struct
initializers. We are using Emacs 27 with cc-mode on Debian 11. I tried 5.35
release, but that hangs. I saw there's a number of fixes so took a look at
Branch_5_35, but that is broken because it's missing defun c-update-new-id.
Therefore, I took cc-mode 5.35.1 from Emacs 28.2 and use it with Emacs 27
with a few minor modifications:

1. Add to cc-defs.el before defconst c-version (this const doesn't exist in
Emacs 27):

     ;; The initial anchoring is for better performance in searching
matches.
     (defconst regexp-unmatchable "\\`a\\`"
       "Standard regexp guaranteed not to match any string at all.")

2. Comment-out declare in cc-bytecomp.el and cc-langs.el (this is an Emacs
28 option):

     ;; (declare (speed -1))


I see that 5.35.1 is slow and on average about 10% slower than 5.34.

Is there any possibility of speeding up cc-mode?

I wrote a small tester to compare paging through a file using
fundamental-mode vs cc-mode. It will find files that take less than a
second with fundamental mode and then many (hundreds) of seconds with
cc-mode. For example, the Linux,
https://github.com/torvalds/linux/blob/master/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
has a literal-mode visit of 0.3 seconds and cc-mode visit of 309.5 seconds
using the tester below. Note, the slowness of Emacs is dependent upon the
number of file you find and then kill. If you time a fresh emacs session
just visiting phy_n.c it will take 18 seconds.

You can try the timing on code such as:

  1. LLVM, https://github.com/llvm/llvm-project/archive/refs/heads/main.zip
  2. GCC, https://github.com/gcc-mirror/gcc/archive/refs/heads/master.zip
  3. Linux, https://github.com/torvalds/linux/archive/refs/heads/master.zip

To time the downloads:

  ccMode=/path/to/byte-compiled-cc-mode
  ccTimer=/path/to/time-cc-mode-file-visit.el
  ccSummary=/path/to/cc-mode-timing-summary.pl

  unzip DOWNLOAD.zip
  cd DOWNLOAD
  emacs -Q -L $ccMode -l $ccTimer '--eval=(progn (time-cc-mode-file-visit)
(kill-emacs))'"
  $ccSummary

This will show you the top 100 slowdowns comparing fundamental-mode to
cc-mode.

Thanks
John

----- time-cc-mode-file-visit.el

;;; -*- lexical-binding: t -*-

(require 'cc-mode)
(require 'subr-x)
(require 'cl-macs)

(defun time-literal-visit (c-file)
  (let ((start-time (current-time))
        (buf (find-file-literally c-file)))
    (while (not (eobp))
      (scroll-up-command))
    (kill-buffer buf)
    ;; literal visit of file took
    (float-time (time-since start-time))))

(defun time-cc-visit (c-file)
  (let ((start-time (current-time))
        (buf (find-file c-file)))
    (while (not (eobp))
      (scroll-up-command))
    (font-lock-ensure (point-min) (point-max))
    (kill-buffer buf)
    ;; cc visit of file took
    (float-time (time-since start-time))))

(defun time-cc-mode-file-visit ()
  "Visit C/C++ files under current dir literally and normally.
During visit, page down though the file till the end.

Creates ./cc-mode-timing.txt"
  (let* ((c-files-str (shell-command-to-string
                       ;; ./cc-mode-timing-summary is created by
cc-mode-timing-summary.pl
                       (concat "find . \\( -name cc-mode-timing-summary
-prune \\) -o "
                               "\\( -type f -regex
'.+\\.[c|h]\\(pp\\|xx\\)?' -print \\) | sort")))
         (c-files (split-string (string-trim c-files-str) "\n"))
         (timing-list-file "cc-mode-timing-files.txt")
         (result-file "cc-mode-timing.txt"))

    (setq enable-local-variables nil)       ;; Don't prompt for file local
emacs variables settings
    (setq scroll-error-top-bottom t)        ;; Let `scroll-up-command'
scroll to bottom of file
    (setq large-file-warning-threshold nil) ;; Don't prompt if file is big

    (write-region c-files-str nil timing-list-file)
    (write-region (format "# Emacs %s\n# c-version %s\n" emacs-version
c-version) nil result-file)
    (cl-loop for c-file in c-files do
             ;; Write incrementally that way we can see where hangs are
             (write-region (concat c-file ":\n") nil result-file 'append)
             (let* ((literal-visit-took (time-literal-visit c-file))
                    cc-visit-took
                    time-diff)
               (write-region (format "   literal visit: %f seconds\n"
literal-visit-took)
                             nil result-file 'append)
               (setq cc-visit-took (time-cc-visit c-file))
               (write-region (format "   cc visit     : %f seconds\n"
cc-visit-took)
                             nil result-file 'append)
               (setq time-diff (abs (- cc-visit-took literal-visit-took)))
               (write-region (format "   time-diff    : %f seconds%s\n"
time-diff
                                     (if (> time-diff 5) " SLOW (>5
seconds)" ""))
                             nil result-file 'append)))))

----- cc-mode-timing-summary.pl -----

#!/bin/perl

use strict;
use warnings;
use Cwd;
use File::Basename;
use File::Copy;
use File::Path qw(make_path remove_tree);

my $nTimings = 100;
my $outDir = "./cc-mode-timing-summary";

if (@ARGV != 0) {
    die "usage: $0
Generates a summary containing the top $nTimings timings from
./cc-mode-timing.txt
which is generated by time-cc-mode-file-visit.el.
";
}

if (-e $outDir) {
    print "$outDir exists. Delete it (y/n) [n]? ";
    my $ans = <STDIN>;
    if ($ans =~ /^y(?:es)?$/) {
        remove_tree($outDir);
    }
}
mkdir($outDir) || die "mkdir $outDir";


my @header;
my %timings;

my $timingFile = "cc-mode-timing.txt";

open(my $fh, $timingFile) || die "read $timingFile: $!";
my $lineNum = 0;

# File has format:
#   # header lines
#   ./path/to/foo.cpp
#      literal visit: 0.186026 seconds
#      cc visit     : 20.080301 seconds
#      time-diff    : 19.894275 seconds SLOW (>5 seconds)
# where the SLOW comment is optional
while (my $line = <$fh>) {
    $lineNum++;
    if ($line =~ /^#/) {
        push(@header, $line);
        next;
    }
    if ($line =~ m{^(\./.+):$}) {
        my $file = $1;
        my $literalTimeLine  = <$fh>; $lineNum++;
        my $ccTimeLine       = <$fh>; $lineNum++;
        my $timeDiffLine     = <$fh>; $lineNum++;
        if ($timeDiffLine !~ /^\s+time-diff\s*:\s*(\d+\.\d+) seconds/) {
            die "$timingFile:$lineNum: unexpected content, expected
'time-diff: N.M seconds'\n";
        }
        my $timeDiff = $1;
        push(@{$timings{"$timeDiff"}}, [$file, $literalTimeLine .
$ccTimeLine . $timeDiffLine]);
    } else {
        die "$timingFile:$lineNum: unexpected content, expecting
./path/to/file.cpp\n";
    }
}
close($fh) || die "close $timingFile: $!";


my @sortedTimingDiffs = sort { $b <=> $a } keys(%timings);
if (@sortedTimingDiffs > $nTimings) {
    splice(@sortedTimingDiffs, $nTimings);
}

my $out = "$outDir/summary.txt";
open(my $outFH, ">", $out) || die "open $out: $!";
print $outFH "# Top $nTimings slowdowns\n";
print $outFH @header;
foreach my $timeDiff (@sortedTimingDiffs) {
    foreach my $timeInfo (@{$timings{$timeDiff}}) {
        my ($file, $fileTimes) = @$timeInfo;
        print $outFH "$file:\n". $fileTimes;
        my $fDir = $outDir . "/" . dirname($file);
        if (! -d $fDir) {
            make_path($fDir);
        }
        my $dst = "$fDir/" . basename($file);
        copy($file, $dst) || die "cp $file $dst: $!";
    }
}
close($outFH) || die "close $out: $!";

print "Created: $outDir\n";