Today, I'll explain the steps to upgrade nodejs on Rocky Linux.
List all the available versions of nodejs
dnf module list nodejs

To stop using the currently used version of nodejs.
dnf module -y reset nodejs
Choose the version you'd like
Since I want to use version 20, I ran
dnf module -y enable nodejs:20
And finally you can install with the following command,
dnf module -y install nodejs:20/common