Are extensions downgrade-safe?

Hi everyone

In PostgreSQL, there are times where a new version of an extension is
included in a minor-version upgrade. If I do a minor-version upgrade of my
PG database, upgrade the version of the extension and then have a need to
revert (downgrade) to the old PG minor version, what should I do with the
extension?

I searched the extensions in contrib and none comes with a downgrade
script. Are extensions that do not come downgrade script downgrade-safe? Or
do I need pg_dump or restore from backup to downgrade safely?

Thanks
Robert