As mentioned above, most good audio interfaces have preamps and phantom power if you ever need to run a condenser microphone. I see that you don't "plan to do" that, but never say never. Condenser mics require an external power source, (phantom power), so having it available is good.
An audio interface with MIDI ports will allow you to connect your keyboard to the interface and never have to worry about some MIDI driver that may need updating periodically. You would have to update the interface drivers now and then. Maybe. My computer is very old and nothing has been updated on it since ~ 2012, including the interface. Works like a charm. But with your new computer, it's best to maintain it for as long as you can, including drivers.
You should be able to record with acceptable latency with a good, modern interface while using amplifier plug-ins such as Amplitube and the like. I can't discern record lag running 10 ms latency, but others can. That's usually a 'safe' number that won't upset the processor. You should be able to do better with your rig.
The ASIO driver for your computer's internal sound chip is not good for recording. Try WASAPI if you have it available until you get an interface with an ASIO driver written for that device. ASIO4All is a wrapper for the computer's WDM/KS driver as rsinger noted, but isn't a real ASIO driver. Some folks have had good luck with it; many do not, including me (15 years ago).
The best, long-term solution for recording satisfaction is with a modern audio interface with an ASIO driver made for it. This will almost guarantee good performance for many years, with updates and bug fixes included.
Some good companies to check out: MOTU, TASCAM, Focusrite, Presonus. Many of the people here will have their gear posted at the bottom of their messages, read a few pages and see what they're using. Ask questions and maybe list here what features you would like or need, although you have already said what you want, mostly.
Welcome to the forum.