Not necessarily.
My JP-8000 gave up the ghost last year. I could probably fix it but instead I decided to put it back into its carry case.
Since upgrading my workstation I don't really find this a problem.
I don't understand this one... Are you tracking to your DAW or to a multi track recorder of some kind? If the former, then sure there is latency involved.
Back in the early 2000's I was entirely hardware with all my instruments going into a conventional mixer and I mixed and monitored from there and finally recorded a stereo mix into my workstation. So, yes, in that situation, zero latency.
Perhaps I am misunderstanding something here...
So, to answer the original question, at this moment in time I have feet in both the hardware and software camp. And you can't get more hardcore hardware synth than one of these:-
No ROM... No presets