I'm sorry to hear you're disappointed with our mandolin sample library!
Hopefully I can give you a little insight on the library that might be helpful. And if the library doesn't seem like it's going to work out for you, please get in touch with me through our support form (there's a direct email address on that page, too).
As you mentioned, there's fret buzz, particularly on the loudest dynamic and on the G string. We try to sample a wide dynamic range, meaning that the loudest dynamic is going to naturally give you that fret buzz from when you really dig into the strings. Any fret buzz on the soft or medium dynamic is not intentional, so if you're running into any fret buzz elsewhere, that's something I'll take a look at for an update to the library.
For more graceful parts, I'd recommend lowering the dynamic curve setting so that it's biased towards the softer dynamics. Admittedly the default velocity scaling should probably put the loudest velocity a little higher so that it doesn't happen so readily.
Overall, it's a delicate balance, making sure to record imperfections like fret noises, buzz, and other extraneous noises are included in order for the library to sound more human, while still allowing you to get purer, more pristine playing when needed. It's my experience that if you sample an instrument too cleanly, it tends to sound robotic and lifeless--which is ironic, because as a player, you strive to eliminate all those little extraneous noises (as you mentioned). Noises like that are what help sample libraries sound more believable, which is one of the reasons we included sampling a misfretted sustain articulation that you can mix into faster passages to help with the realism.
You can change the volume/amount of added pick noise in the SETUP section of the interface, and if you need the releases to sound cleaner and more subtle, you can reduce their volume in the SETUP section as well.
In the future I'd like to add more control over the release samples to allow you to adjust the smoothness of the transition between the sustain and release samples, and possibly a way to limit the length of the release samples, too.
The pick noise actually does track velocity, I just double-checked to be sure. Though as I mentioned before, if you need to reduce the amount of added pick noise, you can do that from the SETUP section of the library's interface. At 100%, it's still audible at lower dynamics, so you might try something like 25-50% instead (or just leave it off, of course).
It all depends on where the fretting position is set to. If you play a note outside of the current fretting position, half a second later (or so), it triggers a position change noise to simulate the player returning to the desired fretting position. If you're playing a lot of notes outside of that fretting position, try moving the fretting position control to that range instead. The fretting position can also be automated to a MIDI CC if you want to control it in real-time. I usually just leave it in one general area, though. And as you noted, you can always disable the automatic fret noises and manually trigger them using the performance effect key for the fret noise, which is mapped above the main playing range.
I tried it on my end and can't reproduce what you're running into. It's also strange that the middle position would have a quiet/brittle sound, since it should sound pretty much just like when the pick modeling is disabled. If you load the library in the standalone Kontakt application (just to eliminate some variables), do you run into the same issue?
We use real samples of hammer-ons, pull offs, and slides. However, I do have some ideas on how to improve their realism through scripting, particularly when it comes to playing trills or depending on the current duration of the note. I'm also open to any recommendations you have, and as I mentioned before, feel free to get in touch with me via email!
There's a resonance amount setting in the SETUP section that should be helpful to dial in how much resonance you want--this feature recreates the sympathetic resonance between strings. There's still going to be body resonance in the articulations themselves, and in some cases possibly a little sympathetic resonance. The engineer we worked with is no stranger to recording mandolin, so muting the string ends and tracking down any other little unwanted vibrations before recording instruments is a routine part of the process.
Anyway, hopefully some of that information is useful to you, and I do appreciate any feedback and suggestions you have, too.