All in the title.
If you are using the default terminal shipped with MSYS2, i.e. mintty, then here the steps.
Moreover, mintty Options... menu will let you customize many other behaviors of the terminal.
Thanks a lot Matthieu. I didn't see this menu, I was trying to find some configuration file somewhere.
No problem, the menu is somewhat hidden a bit when you don't know it.
The settings are stored in a file called .minttyrc in your $HOME directory, if you ever want to fiddle with them :)