Re: Web browser not prompt authentication for guest wifi users (Non domain users)
not sure I understand actions and how they progress in time, from your description.
It might help if you note client's OS, because for example mobile platforms do have captive portal detection mechanisms and if they are not able to connect to, for example, captive.apple.com in Apple iOS, then they will pop-up with auth request in OS default browser.
No windows there might be some background app trying to connect somewhere running under system and so once you get connected it will try to connect and cause auth pop-up.
Also note what kind of config you have, for example RADIUS based captive portal on SSID is just one possible option.
For deeper troubleshooting and help please contact Fortinet TAC (open ticket via support.fortinet.com)