Stateless NAT in nftables with maps for performance

Hi netfilter community!
I'm trying to orchestrate the generation and maintenance of thousands of dnat & snat rules in a stateless configuration within the prerouting and forward hooks with chain types of filter, and unfortunately am hitting a big performance barrier as a result.  Its fine with a few thousand, but once we get into the tens of thousands of rules, things start slowing down linearly, which is why we would like to switch to maps.  However, I cannot figure out 1) if maps are supported in filter chains within prerouting/forward hooks and 2) if it supports the syntax for stateless nat?

For example:

table ip mytable {
	map dnat_map {
		type ipv4_addr : ipv4_addr
	}
	chain dnat {
		type filter hook prerouting priority raw; policy accept;
	}
}

I normally generate stateless dnat's via:

$ nft add rule ip mytable dnat ip daddr 100.101.84.137 counter ip daddr set 10.11.1.32 notrack comment "comment 1"


But, lets say I want to try to replace this rule with a map.  If i populate my map with a a key pair like:

$ nft add element ip mytable dnat_map { 100.101.84.137 : 10.11.33.32 }

How can I represent my original rule using a map?

I've tried:

$ sudo nft add rule ip mytable dnat ip daddr @dnat_map ip daddr set @dnat_map
Error: unknown raw payload base
add rule ip inode dns-dnat ip daddr @dnat_map ip daddr set @dnat_map
                                                           ^^^^^^^^^

and a few other variations, but I cannot figure out if this is even supported.

Any feedback would be great, I'm running nftables v1.1.5 (Commodore Bullmoose #6) & kernel 6.8.0-79-generic x86_64 on ubuntu 24.04

Thanks!